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PREFACE 


This  report  was  created  for  the  F-16  Aircrew  Training  De¬ 
velopment  Project  contract  no.  F02604-79-C8875  for  the  Tactical 
Air  Command  to  comply  with  the  requirements  of  CDRL  no.  B043. 

The  project  entailed  the  design  and  development  of  an  instruc¬ 
tional  system  for  the  F-16  RTU  and  instructor  pilots.  During  the 
course  of  the  project,  a  series  of  development  reports  was  issued 
describing  processes  and  products.  A  list  of  those  reports 
follows  this  page.  The  user  is  referred  to  Report  No.  34,  A 
Users  Guide  to  tne  F-16  Training  Development  Reports,  for  an 
overview  and  explanation  of  the  series,  and  Report  No.  35,  F-16 
Final  Report,  for  an  overview  of  the  Instructional  System 
Development  Project. 
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EXECUTIVE  SUMMARY 


The  media  selection  process  utilized  in  this  program  is 
documented  in  three  separate  reports.  Report  No.  20,  "F-16  Media 
Selection  and  Utilization  Plan  Report",  describes  the  initial  31 
step  procedure  designed  to  select  instructional  media  for  various 
categories  of  objectives.  Report  No.  31,  "F-16  Training  Media 
Mix",  describes  the  15  step  procedure  actually  employed  on  the 
project.  Report  No.  26,  "F-16  Pilot  Media  Selection",  provides 
the  raw  data  used  in  the  selection  process.  Taken  together,  they 
provide  a  detailed  description  of  the  rationales  employed,  pro¬ 
cedures  followed,  data  generated,  recommendations  made,  and  media 
mix  utilized. 

Media  selection  consists  of  assigning  instructional  media  to 
groups  of  objectives  in  such  a  way  as  to  maximize  training  effec¬ 
tiveness  while  minimizing  training  costs.  All  objectives  are 
sorted  into  one  of  two  major  categories:  academic  objectives  and 
hands-on  objectives.  Both  sets  are  then  processed  along  parallel 
channels,  whereby  the  requirements  of  each  objective  are  computer 
matched  to  the  capabilties  of  each  medium.  This  produces  a 
prioritized  list  of  media  to  match  each  objective.  These  objec¬ 
tives  are  then  grouped  into  lessons,  trainer  sessions,  or  sorties 
according  to  commonalities  of  both  content  and  medium.  Later, 
during  the  syllabus  building  process,  these  lessons,  sessions, 
and  sorties  are  sequenced  into  a  functional  curriculum. 
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F-16  TRAINING  MEDIA  MIX 


1.0  INTRODUCTION 


Media  selection  is  a  process  of  minimizing  costs  attendant 
to  media  acquisition,  use,  and  maintenance  while  maximizing 
instructional  effectiveness.  It  is  an  engineering  process  in 
which  the  benefits  which  can  be  realized  through  application  of 
learning  principles  and  instr uc tional  research  are  balanced 
against  the  lessons  of  experience  and  economics  in  designing 
real-world  instructional  systems  and  providing  for  the  necessary 
equipnent  and  irns tr uc t iona  1  materials. 

Media  selection  is  one  stage  of  the  instructional  develop¬ 
ment  process  which  is  partly  adm in i s tra t ive  in  nature  and  partly 
governed  by  the  principles  of  psychology  and  learning.  On  one 
hand  it  uses  factor  optimization  decision-making  techniques  to 
produce  a  "best"  media  decision  in  the  presence  of  conflicting 
goals.  This  resolution  of  the  conflicting  demands  for  instruc¬ 
tional  sophistication ,  economy,  and  ease  of  administration  is 
very  much  an  administrative  concern.  On  the  other  hand  media 
selection  must  taka  into  account  the  creative  application  of  the 
principles  of  instructional  strategy  and  materials  developnent 
that  allow  it  to  happen.  Without  a  judgment  along  this  dimen¬ 
sion,  two  competing  media  selection  plans  must  be  assumed  equally 
effective,  regardless  of  the  sophistication  of  the  media  devices 
chosen  . 

Few  processes  affect  an  instructional  system's  cost  and 
practical  utility  as  profoundly  as  media  selection.  The  develop¬ 
ment  of  an  instructional  system  for  aircrew  training  commonly 
involves  the  expenditure  of  thousands  of  dollars  for  learning 
center  media,  the  expenditure  of  still  other  thousands  of  dollars 
over  the  lifetime  of  the  system  on  the  developnent  and  update  of 
mediated  instruction,  and  multi-millions  of  dollars  for  aircraft 
simulation  training  devices.  Decisions  made  during  the  media 
selection  process  have  a  tremendous  financial  impact. 


1.1  Purpose 

~-*The  purpose  of  this  paper  is  to  outline  the  method  and 
rationale  for  the  selection  of  both  learning  center  and  training 
device  media  for  use  in  the  F-16  instructional  system.  The  F-16 
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ins tr uc t iona 1  developers  have  attempted  to  address  two  main 
problems  which  presently  exist  in  the  media  selection  process: 

( 1)  the  problem  of  striking  a  balance  between  the  instr uctional 
and  economic  factors  which  influence  media  decisions  in  such  a 
way  that  one  kind  does  not  suffer  unduly  at  the  hands  of  the 
other  and  (2)  the  problem  of  systematizing  consideration  of 
these  major  factors  in  the  context  of  an  organized  process. 

Though  the  methodology  reported  nere  was  developed  in  the  context 
of  instructional  system  development  for  a  single-seat  aircraft, 
it  is  felt  that  minor  adaptations  can  render  the  methodology 
usable  for  multi-person  crews  of  any  number  and  non-aircrew 
training  applications  as  well.^ 


1.2  Organization 

The  main  body  of  this  report  is  contained  in  Sections  2.0 
and  3.0.  Section  2.0  is  introductory.  It  outlines  tne  media 
selection  problem  and  relates  the  media  selection  process  to 
other  instructional  developnent  processes.  In  particular,  the 
media  selection  document  is  characterized  as  a  "living"  document 
to  be  maintained  on  a  continual  basis  along  with  several  other 
instructional  system  data  base  documents.  This  maintenance  proce¬ 
dure  insures  that  the  instr uctional  development  principles  built 
into  the  system  during  its  design  remain  there  during  its  opera¬ 
tion.  This  section  also  names  the  set  of  factors  and  constraints 
which  need  to  be  balanced  during  media  selection.  Section  3.0 
describes  the  media  selection  process.  The  F-l<3  media  plan 
resulting  from  the  media  selection  process  described  in  Section 


2.0  RATIONALE 


This  section  relates  tne  media  selection  process  to  other 
development,  practical,  and  economic  concerns.  it  begins  with  a 
statement  of  the  scope  of  media  selection  activities,  describes 
how  media  selection  influences  and  is  influenced  oy  other 
instructional  development  activities,  and  discusses  the  balancing 
factors  and  constraints  which  operate  within  media  selection. 


2.1  The  Scope  of  Media  Selection  Activities 

Media  selection  for  aircrew  training  consists  of  two  main 
categories  of  activity.  The  first  deals  with  selection  of  media 
for  use  in  learning  centers  for  information  and  knowledge  convey¬ 
ance  and  practice.  This  includes  well-known  standard  media  types 
varying  from  the  very  simple  and  inexpensive  (e.g.,  charts,  worx- 
sheets,  and  summary  pages)  to  the  complex  and  costly  (e.g., 
computer-assisted  instruction,  table-top  trainers,  and  videotape 
recording)  .  It  includes  media  which  are  intended  for  use  by 
isolated  students  (e.g.,  workbooks,  worksheets,  and  tutorials), 
students  in  small  groups  (e.g.,  discussion  groups  and  problem¬ 
solving  exercises)  and  students  in  large  groups  (e.g.,  lecture). 

A  second  area  of  media  selection  centers  in  the  selection  or 
configuration  of  simulation  training  devices  which  are  intended 
to  create  to  some  degree  a  real-wo rld-1  ike  performance 
environment  in  which  the  student  may  practice  either  part-task  or 
complete  behaviors.  Examples  in  this  area  also  range  from  tne 
simple  (e.g.,  dead  systems,  mocx-up  panels,  displays),  to  tne 
complex  (e.g.,  operational  flight  trainers,  computer  avionics 
simulations,  weapon  system  trainers)  ,  depending  on  the  scope  of 
the  enviromental  factors  being  simulated. 

There  is  much  uncertainty  and  debate  in  the  training  device 
design  area  of  media  selection  because  of  the  growing  complexity 
of  training  devices  and  simulated  environments  and  the  uncer¬ 
tainty  surrounding  the  actual  instructional  effectiveness  of 
given  optional  configurations.  in  the  past  no  systematic  body  of 
research  data  or  detailed  record  has  been  compiled  to  guide 
developers  in  specifiying  training  device  char ac ter i s t ics ,  and 
the  practice  of  determining  device  cha r ac te r  i  s t ics  in  a  non- 
subjective,  organized  fashion  from  a  statement  of  the  training 
requirements  has  not  been  widely  adopted. 

Because  there  is  a  degree  of  indefiniteness  involved  in  both 
areas  of  media  selection  and  some  difficulty  in  supplying  firm 
answers  to  either  learning  center  or  training  device  media  selec¬ 
tion  questions,  tne  process  of  selection  is  not  generally  as 
precise  an  activity  as  instructional  designers  would  desire. 
Subjective  approaches  to  selection  abound,  although  some  highly 
systematic  methods  have  been  created  (Braby,  et.  al.,  1975).  For 
tne  most  part,  the  problem  faced  by  the  developer  is  a  matter  of 
distinguish i  i  between  necessities  and  nonnecessities. 
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2.2  Media  Selection  in  Relation  to  Other  Instructional 

Development  Processes 

While  earlier  stages  in  the  instructional  development 
process  are  analytic  phases  which  gather  and  correlate  data  for 
use  by  the  developer,  media  selection  is  a  synthetic,  or  design, 
stage  in  which  plans  are  formulated  which  jive  structure  to  the 
instructional  system.  Media  selection  builds  on  the  products  of 
previous  developnent  stages,  and  takes  place  in terac t i vel y  with 
other  development  tasks,  therefore  it  is  appropriate  to  describe 
its  relation  to  other  stages  of  instructional  development. 

Early  stages  of  i  ns  tr  uc  t  ional  development  supply  the 
substance  which  the  media  selection  process  works  with.  Media 
selection  for  F-15  occurs  following:  (1)  the  generation  of  a  job 
task  inventory  or  task  listing,  (2)  selection  of  a  set  of 
terminal  training  tasks,  (3)  generation  of  a  list  of  difficult- 
to-o pera t ional i ze  behaviors  called  "goals",  (4)  generation  of 
c r  i  ter  ion- r ef  enced  objectives  (CROs)  to  be  used  as  terminal 
performance  measurement  points,  and  (5)  generation  of  objectives 
hierarchies.  The  results  of  these  processes  taken  together  form 
a  data  base  of  instructional  requirements  for  F-16  pilots  (and 
also  for  F-16  instructor  pilots,  since  the  same  data  base  was 
created  for  that  position  as  well).  The  data  base  documents  are 
used  as  inputs  to  the  media  selection  process.  They  list  the 
instructional  events  for  which  media  needed  to  be  selected. 

Those  foundation  documents  and  the  details  of  their  generation 
are  described  in  separate  F-16  project  reports. 

Concurrent  with  media  selection  and  interacting  with  it,  a 
process  of  syllaous-build ing  or  "sequencing"  takes  place.  For 
sequencing,  the  same  base  of  docunentatioa  from  previous  stages 
is  used  as  an  input.  Sequencing  is  the  process  of  arranging 
those  instructional  events  taken  from  the  CROs  and  objectives 
hierarchies  i r  the  order  students  encounter  them  in  the  course  of 
instruction.  Ideally,  the  sequence  (syllabus)  promotes  the  best 
obtainable  gridual  progression  of  student  behavior  from  the  time 
of  entry  into  training  to  the  point  of  mastery  of  all  terminal 
course  behaviors.  Media  selection  and  sequencing  are  closely 
related  to  each  other.  The  order  of  instructional  events  in 
training  often  makes  particular  media  selections  more  practical. 
Media  selection  decisions  determine  the  types  of  instructional 
events  and  environments  which  can  be  used  in  the  sequence. 

Special  attention  in  both  processes  is  given  to  adjusting  the 
student's  ins tr uc t ional  experience  to  within  certain  parameters 
wnich  the  student  will  find  tolerable  or  agreeable  in  terms  of 
variety,  pace,  redundancy,  and  practical  usability. 

The  results  of  the  selection  and  sequencing  processes  are  a 
course  syllabus  and  a  list  of  individual  media  assignments  by 
instr uc tional  event.  These  documents  lead  to  further  instruc¬ 
tional  system  development  activities,  although  they  represent  by 
themselves  a  significant  addition  to  the  fundamental  data  base 
documentation  referred  to  earlier.  Throughout  the  lifetime  of 
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the  instructional  system  these  documents  should  have  a  life  of 
their  own,  changing  as  the  documentation  which  acts  as  input  to 
them  changes  (i.e.,  the  task  listing,  goal  analysis,  CROs,  objec¬ 
tives  hierarcnies)  and  in  turn  influencing  other  documents.  An 
instructional  system  which  does  not  maintain  these  documents, 
though  derived  through  the  application  of  good  instructional 
development  practices,  cannot  remain  functional  and  efficient  for 
long  . 

The  aiedia  selection  document,  taken  in  conjunction  with  the 
syllabus  serves  several  purposes.  It  serves  as  an  inventory  base 
during  the  actual  production  of  instructional  materials.  Prior 
to  that  it  serves  as  a  base  for  calculating  projections  of  the 
production  organization  and  costs  which  will  be  required  during 
production  planning.  For  instance,  the  types  and  numbers  of 
instructional  presentations  (called  "segments")  can  be  determined 
only  after  media  selection,  as  well  as  projection  of  the  number 
and  types  of  media  devices  required  to  present  instruction  for  a 
given  volume  of  students.  Indirectly,  these  figures  lead  to  the 
partial  projection  of  numbers  and  types  of  instructors,  adminis¬ 
trators,  and  technicians  who  will  be  required  to  staff  the 
student  Learning  Center,  simulator  locations,  media  equipment 
repair  facilities,  and  media  revision,  production,  and  reproduc¬ 
tion  facilities.  All  of  these  items  have  both  short-  and  long¬ 
term  financial  implications.  In  fact,  the  financial  impact  of 
these  decisions  is  the  major  cost  factor  in  the  instructional 
system  operation. 


2.3  Factors  in  Media  Selection 

In  minimizing  the  costs  and  maximizing  the  benefits  of  media 
selections,  many  factors  must  be  considered.  Media  selections 
must  be  a  "best"  solution  based  upon  stated  criteria,  the  optimi¬ 
zation  on  a  set  of  sometimes  conflicting  demands.  "Make  it  inex¬ 
pensive"  often  leads  to  different  solutions  than  "make  it  effec¬ 
tive,"  and  frequently  both  disagree  with  "make  it  interesting." 
Ideally,  the  media  selection  process  insures  that  each  selection 
factor  is  properly  considered  in  the  decision-making  process. 

The  media  selection  process  described  in  Section  3.0  of  this 
report  is  structured  in  such  a  way  that  consideration  of  these 
demands  takes  place  in  an  orderly  fashion  and  seeks  to  insure 
that  each  factor  is  considered  in  proportion  to  its  relative 
importance  to  the  development  task  at  hand.  Failure  to  consider 
all  relevant  factors  in  media  selection,  or  undue  and  dispropor¬ 
tionate  emphasis  on  one  factor  can  lead  to  wasteful  spending  or 
can  produce  media  systems  which  are  unpalatable  to  the  student, 
difficult  to  use,  ins tr uc t iona 1 1 y  ineffective,  or  any  combination 
of  these.  Plentiful  examples  of  each  can  be  found  in  the  history 
of  instructional  development. 

The  major  factors  which  have  been  considered  during  the  F-16 
media  selection  process  are  listed  below  along  with  a  statement 
of  the  relative  priority  of  each  within  the  F-16  context. 
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2.3.1  Ability  to  Implement  Effective  Instructional 
Strategies 

The  ability  of  a  device  to  implement  a  specific  instruc¬ 
tional  strategy  is  a  factor  of  media  selection  too  often 
diminished  in  importance,  even  though  it  should  logically  rival 
or  even  outweigh  cost  in  determining  which  devices  are  selected. 
Media  are  preferred  if  they  demonstrate  a  capability  for 
implementing  these  strategies,  and  possible  media  combinations 
and  reconfigurations  are  considered  where  strategies  can  not  be 
accommodated  in  an  attempt  to  produce  media  as  capable  as 
possible  in  implementing  them.  Characteristics  of  a  media  device 
which  take  part  in  the  implementation  of  strategies  are:  (1)  the 

ability  to  generate  appropriate  stimulus  conditions,  (2)  the 
ability  to  receive  student  responses,  (3)  the  ability  to  process 
those  responses,  and  (4)  the  ability  to  provide  appropriate 
counter-responses  (feedback  and/or  subsequent  stimulus  condi¬ 
tions)  .  Depending  upon  the  strategy  being  used,  these  require¬ 
ments  may  be  simple  or  complex.  They  may  call  for  the  ability  to 
present  simple  pictorial,  detailed  photographic,  or  complex 
moving  computer-generated  graphics  as  stimuli.  They  may  call  for 
the  reception  of  responses  from  the  student  which  are  simple  yes/ 
no  answers,  or  complex  verbal  expressions,  or  coordinated  manipu¬ 
lation  of  aircraft-like  controls.  They  may  call  for  responses 
processing  and  feedback  which  ranges  from  simple  correct  answer 
comparisons  followed  by  knowledge  of  results,  to  complex  comput¬ 
erized  calculation  routines  followed  by  simulated  aircraft 
responses.  In  addition,  all  of  these  characteristics  may  be 
required  under  timed  conditions  and  at  varying  levels  of  fidelity 
to  real-world  performance  environments. 

2.3.2  Cost 

The  cost  factor  is  of  extreme  importance  in  media  selections 
because  cost  is  most  often  the  only  upward  limiting  factor.  This 
is  true  of  the  F-16  project.  Media  selections  entail  not  only 
immediate  costs  but  delayed  costs,  some  of  which  continue 
throughout  the  lifetime  of  the  media  system. 

Figure  1  summarizes  the  costs  associated  with  media  device 
procurement  and  operation.  The  table  identifies  media  costs 
during  five  main  time  frames:  (1)  during  the  design  of  media 
systems  and  instructional  materials,  (2)  during  the  procurement 
of  a  media  system,  (3)  during  development  of  the  media  system  and 
instructional  materials,  (4)  during  initial  implementation  of  the 
system,  and  (5)  during  ongoing  use  of  the  media  system  and 
materials.  It  identifies  major  categories  also,  under  which 
expenses  might  occur  during  each  phase. 

Media  vary  widely  in  the  exact  values  of  each  entry  in  this 
table.  Even  for  the  same  medium,  expenses  vary  with  differing 
training  conditions,  such  as  already-available  resources,  differ¬ 
ent  geographic  locations,  differences  in  manpower  availability. 
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and  differing  o r g an i za t iona 1  patterns.  Not  all  media  require  all 
of  the  listed  expenses,  but  all  media  will  have  a  suDset  of  the 
expenses.  The  more  sophisticated  the  medium,  the  more  expenses 
are  likely  to  be  attached,  in  both  the  short  and  the  long  terms. 
For  learning  center  media  the  acquisition  costs  are  compare t ively 
minor,  but  the  maintenance  costs  for  instructional  materials  can 
be  substantial  for  aircrew  traininj.  For  training  devices, 
acquisition  costs  are  often  very  large,  even  more  so  wnen 
multiple  training  sites  and  therefore  multiple  copies  of  a  device 
are  called  for. 

2.3.3  Attention-  and  Interest-Generating  Qualities 

Since  meJia  devices  and  the  presentations  maJe  u  y  them 
represent  a  main  communication  link  during  instruction,  it  is 
important  not  only  tnat  the  message  be  clearly  stated  out  that 
the  presentation  arouse  attention  and  interest  in  the  user. 
Inherent  in  most  ins tr uc t ional  strategies  is  an  interaction 
between  student  and  device  that  requires  active  par  tic ipa t ion  by 
the  student.  This  requirement  for  interaction  can  serve  as  an 
a ttention-genera ting  tool.  In  addition,  if  the  mode  of  interac¬ 
tion  is  interesting  to  the  student  or  facilitates  a  student's 
personal  learning  goal,  it  can  act  as  an  interest-generating  tool 
as  well. 

Many  med ia  selection  decisions  are  made  witnout  considering 
the  impact  of  the  decision  upon  student  attention  and  interest. 
Still  others  are  made  upon  incorrect  assumptions  of  what  tne 
developer  thinks  will  be  attention-getting  or  interesting  to  the 
student  rather  than  upon  data  from  student  trials.  Results  of 
such  selections  are  media  selections  whicn:  (1)  offend  students 
because  they  are  inappropr ia te  to  student's  taste  and  learning 
styles,  (2)  bora  students  because  they  operate  too  slowly, 
because  the  student  has  no  opportunity  to  interact,  or  because 
there  is  too  much  use  made  of  one  medium,  (3)  are  a  distraction 
to  students  because  tney  require  too  much  attention  just  for 
operation,  or  (4)  are  too  fast  for  students'  learning  pace  and 
cannot  be  controlled  by  the  student,  thus  forcing  tne  student  to 
move  at  the  presentation's  rate  or  repeat  the  instruct  ion- -wh ion 
some  media  do  not  allow. 

2.3.4  Use  of  Standard  Systems 

Several  potential  benefits  encourage  utilization  of  exist¬ 
ing,  standard  media  systems  where  possible.  Procurement  of 
devices,  maintenance  service,  acquisition  of  spare  parts,  and 
acquisition  of  replacement  devices  is  facilitated  when  the  media 
system  being  dealt  with  is  well-known  and  wid e 1 y-a v a i 1 abl e . 
Moreover,  for  the  production  and  reproduction  of  media  presenta¬ 
tions  for  such  standard  systems,  specialized  equipment  and 
trained  service  personnel  are  much  more  likely  to  be  available 
than  for  an  exotic  and  little-used  medium. 
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2.4  Constraints  Upon  Media  Selection 


Jnqoverned  by  real-world  constraints,  the  media  selection 
process  can  produce  interesting  and  effective  but  practically 
unusable  solutions.  Resource  and  personnel  availability  levels, 
costs,  and  personnel  capability  are  all  constraints  which  exist 
in  nearly  all  media  selection  activities.  Many  other  constraints 
exist  as  well  due  to  policy,  regulations,  or  limited  supply  of 
some  critical  resource.  Some  are  nard  constraints  which  cannot 
be  changed,  and  some  are  soft.  With  the  proper  j usti f ication 
many  can  be  modified. 

Certain  constraints  existed  at  the  onset  of  the  F-16  media 
selection.  Some  of  these  were  due  to  the  late  timing  of  the 
contractor's  involvement  in  the  F-16  development  effort,  and 
otiiers  were  due  to  general  USAF  policies  or  programs.  This  sec¬ 
tion  contains  a  s.  idy  of  the  F-16  constraints  related  to  media 
selection.  Those  listed  are  a  suoset  of  the  constraints  active 
wi tn  the  context  of  the  F-16  project.  The  total  list  of  con¬ 
straints  and  developer  response  to  them,  is  contained  in  project 
report  no.  15,  "  Pr  o  j  ram/Sy  ste  n  Constraints  Analysis  Report." 

2.4.1  Directives,  Regulations,  and  Manuals 

Of  the  numerous  manuals  tnat  impact  F-16  aircrew  training, 
four  are  particularly  relevant  to  the  media  selection  process: 
TACM  51-50,  TACR  50-31,  AFR  60-1,  and  TACM  51-5.  TACM  51-50 
establishes  minimum  Air  Force  standards  for  traininj  and  qual  i- 
fying  personnel  performing  duties  in  TAG  assigned  aircraft. 
Ciianges  to  tnis  document  may  result  in  the  addition,  deletion,  or 
codification  of  traininj  tasxs,  resuliinj  in  a  subsequent  cnanje 
in  the  media  required  to  teacn  those  tasxs.  An  increase  in  TACM 
51-50  standards  could,  for  example,  increase  tne  requirement  for 
aircraft  and  simulator  time,  while  a  decrease  could  reduce  that 
requirement . 

TACR  50-31  estublisnes  a  program  for  completing,  maintain¬ 
ing,  and  disposing  of  traininj  records  generated  in  TAC  units 
conducting  syllabus  traininj.  More  importantly,  it  provides 
academic  testing  procedures  as  well  as  simulator  and  flying 
performance  evaluation,  course  training  standards,  and  direction. 
Changes  to  the  simulator  and  flying  performance  evaluation  proce¬ 
dures  nay  impact  the  use  of  simulators  and  aircraft  for  training 
purposes  by  altering  the  amount  of  training  device  time  allocated 
to  testing.  AFR  60-1  establishes  policy  for  tne  managenent  of 
Air  Force  flying  resources,  and  guidance  applicable  to  the 
administration  of  flight  management,  lircrew  training,  and 
aircrew  evaluation  programs.  Changes  to  this  document  could 
alter  training  tasks,  i ns t r uc t iona l  proeeduies,  or  testing 
criteria,  all  of  which  could  impact  the  demands  made  upon 
instr  uc  t  iona  1  media. 

TACM  51-5  estublisnes  TAC  policy  on  aircrew  training, 
program  management  criteria,  and  administrative  practices,  and 


outlines  all  aircrew  ground  training  required  to  maintain 
aircrews  at  a  high  state  of  combat  readiness.  Changes  to  this 
document  may  alter  training  tasks,  resulting  in  a  subsequent 
change  in  the  media  required  to  teach  those  tasks. 

2.4.2  Aircraft  Availability 

The  F-16  aircraft  is  clearly  the  most  essential  training 
medium  utilized  by  the  system  and  its  availability  is  critical. 
Ground  based  training  limitations  mandate  that  certain  tasks  be 
taught  in  the  aircraft,  while  the  early-on  unavailability  of 
sophisticated  simulators  places  even  greater  demands  on  aircraft 
availability.  The  primary  constraints  of  aircraft  availability 
are  as  folio ws: 

1.  Delivery  schedule  delays,  such  as  those  currently  projected 
for  F-16  jet  engines,  may  force  some  tasks  to  be  trained 
using  other  media,  or  omitted  from  the  training  syllabus 
entirely . 

2.  Tne  mix  of  F-16A  and  F-16B  model  aircraft  impacts  media 
selection  by  constraining  the  tasks  tnat  may  be  assigned  to 
the  F-16B  aircraft.  The  current  two  F-16B  to  one  F-16A 
ratio  imposes  no  substantial  limitation,  provided  the 
delivery  schedule  delays  discussed  in  (1)  above  do  not 
effect  delivery  of  the  two  aircraft  models  differentially. 
This  could  limit  the  number  of  F-16B  aircraft  available  for 
the  numerous  early  flight  tasks  that  require  A/B  model 

a  ircraf t . 

3.  Developmental  changes  to  the  aircraft  conf i g ur a t ion--ha  rd- 
ware  and  software — alter  pilot  tasks  and  may  render  some 
simulation  training  media  outmoded.  Tnis  characteristic  of 
emerging  weapon  systems  necessitates  a  fine-grained  media 
selection  procedure  which  incorporates  a  regular  review  of 
aircraft  configuration  and  procedures. 

4.  The  post  program,  with  its  associated  source  factors,  will 
have  a  severe  impact  on  aircraft  availabiity  if  it  is  imple¬ 
mented  in  the  F-16  RTU.  Training  tasks  will  have  to  be 
reassigned  to  alternate  media  in  order  to  take  full  advan¬ 
tage  of  the  period  of  high  aircraft  use,  while  allowing 
training  to  continue  during  the  period  of  low  aircraft  use. 

2.4.3  Programmed  Plying  Training 

The  programmed  flying  training  plan  provides  basic  informa¬ 
tion  on  student  flow  and  sortie  generation  rates.  Changes  to 
this  doucment  affect  media  selection  primarily  in  relation  to  th 
number  of  media  devices  required  to  support  a  given  level  of 
student  throughput.  Low  generation  rates  will  necessitate  some 
tasks  being  trained  in  simulators  rather  than  in  the  aircraft, 
while  high  generation  rates  will  permit  tasks  to  be  shifted  from 
the  simulator  to  tne  aircraft. 


2.4.4  USAP  Manning  Constraints 

The  number  of  instructor  personnel  available  for  F-16  train¬ 
ing  has  a  profound  impact  on  the  assignment  of  media  to  various 
training  tasks.  When  instructors  are  available  in  large  numbers, 
seminars,  lectures,  tutorial  sessions,  and  other  instructor- 
dependent  media  may  be  assigned  liberally.  When  these  individ¬ 
uals  are  in  short  supply,  however,  many  objectives  that  would 
otherwise  be  taught  througn  instructor  interaction  are  relegated 
to  student  self-study  media,  such  as  workbooks,  tape/slides  and 
v  ideo  tapes  . 

Changes  to  the  instructor  manning  levels  may  be  precipitated 
by  changes  to  TACM  25-5,  which  contains  procedures  for  computing 
and  publishing  aircrew,  instructor,  aircraft,  and  student 
requirements.  this  standard  applies  to  all  CCT  and  related 
training  units  within  TAC. 

2.4.5  Environmental  Constraints 

Environmental  constraints  refer  to  the  character istics  of 
physical  location,  such  as  weather,  altitude,  geography,  and 
airspace  limitations,  that  limit  the  ability  of  F-16  aircraft  to 
fly  whenever  and  wherever  they  choose.  These  constraints  affect 
only  the  aircraft,  as  all  other  training  media  are  indoor 
resources  and  consequently  unaffected  by  outdoor  conditions. 

1.  TACR  55-16  establishes  standard  operational  and  weapons 
employment  procedures  for  all  tactical  aircrews  operating 
USAF  F-16  aircraft.  Each  operational  unit  affected  by  TACR 
55-16  then  develops  a  supplement  which  is  unique  to  its  own 
location.  For  example,  the  388th  TFW  supplement,  Chapter 
VIII  of  TACR  55-16,  sets  forth  requirements  and  procedures 
to  follow  for  Hill  AFB  which  are  unique  to  Hill  due  to  the 
altitude  of  the  base  and  the  summer  temperatures ,  winter 
flying  conditions,  crosswind  conditions,  and  so  on.  These 
differing  conditions  affect  the  amount  of  simulator  practice 
that  should  be  completed  before  actual  aircraft  task  perfor¬ 
mance,  as  well  as  affecting  the  performance  criteria  asso¬ 
ciated  with  simulator  and  aircraft  training  objectives. 

2.  Conditions  of  weather  variability  will  exist  between  the 
various  F-16  training  sites,  resulting  in  a  variation  in  the 
number  and  quality  of  flying  training  days  available.  This 
may  require  flexibility  in  the  selection  of  media  for  flying 
training  tasks,  such  that  simulator  exercises  may  replace  or 
augment  training  flights  that  have  been  postponed  or 
cancelled  because  of  weather  conditions. 

3.  It  is  assumed  that  local  availability  and  configuration  of 
operating  ranges  (ACMR/I  ava  i  1  ab  i  1  i  t  y)  will  vary  from  one 
training  site  to  the  next.  This  variability  will  apply  to 
operational  air  space  as  well.  Media  selection  should 
encourage  the  maximum  use  of  these  resources  wnere  they  are 
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available,  while  ensuring  maximum  flexibility  for  those 

sites  where  such  resources  are  either  unavailable  or  limited 

to  temporary  TOY  experience. 

2.4.6  Safety 

Safety  is  the  primary  design  factor  in  virtually  all  phases 
of  F-16  aircrew  training,  and  media  selection  is  no  exception. 

The  student  must  be  given  adequate  experience  in  the  relatively 
safe  media,  the  simulators  and  F-16B  model  aircraft,  before  being 
allowed  to  operate  the  F-16A.  AFM  127-1  provides  guidance  for 
the  prevention  and  investigation  of  aircraft  accidents  and 
incidents,  and  all  design  changes  will  be  compared  to  existing 
AFM  127-1  regulations  for  conformity. 

2.4.7  Training  Device  Availability 

The  coordination  of  instructional  development  decision¬ 
making  with  the  logistic:  .  procuring  complex  media  devices,  for 
instance  simulator  equ'pment,  has  been  difficult  in  the  past  as 
mentioned  in  the  previous  section.  Often  decisions  on  such 
devices  take  place  be.'ore  the  arrival  of  instructional  systems 
development  personnel.  Alternately,  such  decisions  are  often 
made  by  developers  ac  ;pg  with  inadequate  information  or  analysis 
upon  whicn  to  base  t\.-  5  recommendation.  The  F-16  project  has 
involved  ins tr uc t ' anal  developers  at  a  very  early  pnase  of  air¬ 
craft  system  design  and  implementation .  Nonetheless,  the  dead¬ 
lines  for  procuring  simulators  came  earlier,  and  many  features  of 
the  F-16  0FT/WS1,  Cr'T,  and  EPT  were  determined  in  advance  of 
developer  arrival. 

A  provision  of  the  F-16  contract  requires  input  from  the 
instructional  developer  on  the  qualities  and  capabilities  desired 
in  the  F-16  OFT.  The  formal  portion  of  this  input  is  contained 
in  project  report  no.  22,  "Recommendations  for  F-16  Operational 
Flight  Trainer  (OFT)  Improvements".  Though  it  nas  been  found 
that  sane  of  the  recommendations  can  be  implemented  in  the  final 
simulator  system  for  F-16,  many  of  them  cannot  because  of  the 
advanced  stage  reached  in  planning  when  the  recommendations  were 
produced.  To  the  extent  tnat  the  design  of  the  simulator  cannot 
be  flexible  to  the  developer's  plans,  a  constraint  is  encountered 
which  the  developer  must  circumvent  by  adapting  the  instructional 
technique  to  the  existing  trainer  ca pab i 1 i t ies . 

Delivery  deadlines  for  already-ordered  training  devices  fall 
after  the  beginning  of  training  of  F-16  pilots.  Availability  of 
the  OFT  is  likely  to  be  four  to  six  months  following  the  start  of 
training.  Availability  of  the  first  CFT  is  scheduled  for  January 
1979.  The  Egress  Procedures  Trainer  is  to  be  installed  and 
working  by  January  1979.  The  interim  training  course  will  begin 
in  January  1979. 

Interim  simulation  capabilities  of  various  types  have  been 
proposed  by  F-16  agencies.  Each  has  attached  to  it  a  set  of 
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constraints  which  will  have  major  impact  on  F-16  training,  parti¬ 
cularly  in  the  area  of  the  syllabus  sequence.  Simulation  capa¬ 
bilities  operating  at  a  distance  from  the  central  training  site 
require  extracts  of  the  syllabus  to  be  concentrated  together  for 
student  work  as  the  simulator  becomes  available.  Moreover, 
instructors  working  at  a  distance  from  the  main  training  site  are 
likely  to  encounter  communication  problems  with  the  main  site. 
Students  may  also  encounter  the  need  for  refresher  instruction, 
depending  upon  the  lag  in  simulator  availability.  Under  these 
circumstances  special  configurations  of  the  instructional  materi¬ 
als,  syllabus,  and  the  instructor  training  and  monitoring  system 
may  be  called  for. 

Simulation  capabilities  operating  at  the  main  site  but  at  a 
lesser  degree  of  sophistication  also  require  special  interim 
syllabi  and  provision  for  later  update  of  student  capability  as 
the  f ul 1-capabi 1 i ty  simulator  becomes  available. 

2.4.8  Ground-Based  Training  Limitations 

Ground-based  training  limitations  are  defined  by  those 
training  tasks  that  cannot  be  adequately  taught  on  the  ground. 
These  limitations  are  of  two  primary  types:  Performance  fidelity 
and  subjective  fidelity.  Performance  fidelity  refers  to  the 
ability  of  a  trainer  or  simulator  to  respond  precisely  as  would 
the  actual  aircraft  in  the  same  siutation.  In  spite  of  the 
incredibly  realistic  simulation  provided  by  some  training 
devices,  they  still  are  unable  to  provide  the  full  range  of 
stimulus  and  response  cues,  especially  in  the  area  of  cockpit 
visuals  and  the  sensation  of  varying  increments  of  forward 
velocity.  Subjective  fidelity  refers  to  the  inability  of  the 
individual  student  to  forget  that  he  is  in  a  simulator  on  the 
ground  instead  of  in  an  actual  aircraft  at  altitude. 

2.4.9  Preliminary  Instructional  Content 

Because  F-16  is  a  highly  complex  emerging  weapon  system,  it 
can  be  expected  that  numerous  changes  will  be  made  to  the  air¬ 
craft  and  to  the  procedures  for  its  operational  use.  Changes  in 
instructional  content  are  dictated  by  these  system  and  procedural 
changes  that  in  turn  dictate  a  change  in  instructional  materials. 
Since  changes  will  be  constant,  this  indicates  a  constraint  will 
be  placed  upon  media  presentations.  It  means  they  must  be 
economical  to  revise,  not  only  in  turnaround  time  and  personnel 
time,  but  in  terms  of  special  equipment  or  technical  ability 
required  to  complete  revisions. 

2.4.10  Requirement  for  Premature  Media  Decisions 

The  difficulty  of  coordinating  instructional  development 
schedules  with  procurement  schedules  sometimes  affects  learning 
center  media  and  simulator  device  decision-making.  To  allow  the 
construction  and  equipping  of  learning  center  facilities  to 
proceed  on  an  operationally  tolerable  schedule,  the  numbers  and 
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types  of  media  devices  and  the  archi tectural  plan  of  the  learning 
center  for  F-16's  first  training  site  had  to  be  determined  and 
announced  before  the  media  selection  process  was  executed.  These 
data  were  provided  wnen  requested  and  formed  the  basis  for  order¬ 
ing  learning  center  equipment  and  specifying  learning  center  lay¬ 
out.  Data  are  not  available  on  whether  the  decisions  were 
correct.  The  requirement  for  an  early  announcement,  however, 
with  the  knowledge  that  the  figures  provided  would  be  used  to 
order  media  equipment,  placed  a  constraint  upon  learning  center 
media  selections  actually  resulting.  It  is  likely  that  the  types 
of  devices  selected  in  that  early  selection  were  correct.  It  is 
not  likely,  however,  that  the  numbers  of  media  devices  was  as 
accurate  in  the  light  of  con t inuall y-r ev i sed  student  load 
figures,  actual  media  selections,  and  updates  to  critical  task 
listing  and  objectives  hierarchy  documentation. 

2.4.11  Absence  of  Funding  for  Additional  Equipment 

Despite  the  presence  of  previously-made  media  decisions,  the 
possibility  that  media  studies  during  instructional  development 
might  reveal  additional  desirable  training  devices  was  not  ruled 
out.  Such  devices  might  include  extra  copies  of  a  1  read  y-ord  er  ed 
devices,  modified  already-ordered  devices,  or  totally  new 
devices.  Should  any  of  these  prove  desirable  and  funding  for 
them  not  be  found,  that  lack  of  funding  would  require  revision  of 
media  and  syllabus  plans  and  would  constitute  a  limitation.  To 
the  extent  that  recommendations  for  changes  and  additions  to  the 
training  device  inventory  are  couched  in  cost  and  effectiveness 
terms,  a  decision  should  be  possible  on  the  desirability  of 
modifying  the  cost  constraints. 

2.4.12  Available  Media  Production  Facilities 

The  ready  access  of  some  types  of  media  production  capabil¬ 
ity  and  the  difficulty  of  obtaining  expertise  and  equipnent  for 
other  types  consitutes  a  constraint  upon  certain  F-16  media 
selections.  The  instructional  materials  created  for  F-16  must  be 
maintained,  updated,  and  replaced  as  required.  Choosing  media 
for  which  production  facilities  and  personnel  were  difficult  to 
obtain  or  train  would  be  wasteful. 
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3.0  MEDIA  SELECTION  PROCESS 


This  section  contains  a  step-by-step  description  of  the 
media  selection  model  used  for  F-16.  The  model  contains  a  set  of 
procedures  along  with  the  documentation  aids  necessary  to  support 
the  decision-making  process  and  record  decisions  made.  The 
description  which  follows  describes  each  step  in  the  process 
separately  at  a  level  of  detail  that  should  enable  the  media 
selection  process  to  be  performed  by  personnel  other  than  those 
originating  the  process. 

A  main  feature  of  the  model  is  its  division  into  two  main 
activities  (1)  selection  of  academic  or  learning  center  media, 
and  (2)  selecting  of  s im ul a  to r- type  training  devices.  Learning 
center  media  devices  are  oriented  toward  the  imparting  of 
information,  the  formation  of  new  concepts,  the  learning  and 
practice  of  new  rules,  and  other  cognitive  exercises.  The 
amounts  of  engineering  involved  in  their  selection  is  minimal. 

On  the  other  hand,  training  devices  which  are  not  of  this  group, 
are  not  prepared  mainly  for  the  establishment  of  the  conceptual 
and  informational  base  but  for  the  practice  of  actual  job  1 

behaviors  or  behavior  fragments  in  simulated  job  env  ironments . 

These  devices  may  be  selected  from  a  predetermined  set  of  catego¬ 
ries,  or  be  engineered  individually  to  create  this  job-related 
environment . 

The  general  limitations  in  media  selection  methodology 
described  in  Section  1.0  of  this  report  apply  to  both  the  learn¬ 
ing  center  media  selection  and  the  training  device  selection.  At 
present  many  questions  of  media  selection  have  no  firm  answers. 

For  the  selection  of  learning  ccenter  media,  few  firm  guidelines 
are  provided  to  developers  other  than  the  rationale  each  one  can 
establish.  The  selection  of  s imul a  tor- type  training  devices 
requires  a  body  of  coherent  and  consistent  procedures  for 
deriving  training  device  specifications  from  task  statements, 
such  procedures  are  currently  evolving  but  are  by  no  means 
complete.  The  methodology  in  this  report  parallels  the  learning 
center  media  selection  process.  Training  device  design 
specifications  are  not  addressed  in  this  report. 

Inherent  in  the  procedures  described  in  this  report  are 
certain  assumptions: 

1.  Learning  center  media  should  be  selected  with  due  pro-  ; 

portional  emphasis  upon  the  instructional  strategy  to  be  i 

used  during  instruction.  Several  media  selection  models 
acknowledge  this  principle  and  attempt  to  define  the 
elements  of  instructional  strategy  which  figure  in  media 
decisions.  For  tne  purposes  of  this  report,  a  perspective 
of  instructional  strategy  is  taken  which  examines  the 
individual  instructional  displays  to  be  received  by  the 
student,  classifies  them,  and  utilizes  systematic  rules  for 
their  delivery  to  the  student.  This  assumption  is  necessary  ; 
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to  avoid  the  evils  of  a  cos  t-  f  ac  tor  s-onl  y  media  selection 
model  and  to  give  balance  to  media  selection  by  considering 
more  of  the  factors  relevant  to  media  selection. 

2.  Alternate  media  selections  produced  by  this  model  are 
assumed  to  be  equivalent  in  instructional  effectiveness. 

This  is  a  difficult  assumption  which  virtually  all 
developers  reject  on  the  basis  of  intuition.  Given  the 
present  state  of  the  art  in  measuring  instructional 
effectiveness,  however,  and  the  inability  of  instructional 
psychologists  to  produce  any  scale  measuring  units  of 
effectiveness  for  comparison  purposes,  this  assumption  is 
necessary.  All  media  selection  models  which  take  cost  into 
account  must  make  this  assumption  or  else  account  for 
variable  effectiveness  through  an  estimation  method. 

3.  The  learning  center  and  training  device  selection  and  design 
process  can  be  improved  through  a  process  of  validation 
which  compares  intended  with  actual  outcomes  in  student 
performance.  Though  this  sounds  reasonable  and  desirable, 
the  requisite  research  and  real-world  effectiveness  studies 
have  not  been  performed  which  supply  the  necessary  data 
base.  Cost  figures  and  effectiveness  indices  are  required 
which  can  be  used  as  inputs  to  the  selection  and  design 
process  and  relieve  the  best-guessing  which  otherwise  must 
taka  place.  The  aoility  to  select  and  design  media  through 
a  process  which  involves  considerations  of  cost  and 
effectiveness  requires  either  a  beginning  data  base  from 
which  the  figures  can  be  derived  or  else  an  initial  set  of 
estimates  which  can  be  validated  through  experience.  That 
data  base  does  not  presently  exist  for  aircrew  training 
devices  and  must  be  built  before  more  precise  trade-off 
studies  can  be  made.  This  is  emphasized  as  a  critically 
important  point  if  progress  is  to  be  made  in  defining  the 
technology  of  media  device  design  and  selection.  The 
ability  to  do  this  further  assumes:  (1)  The  existence  of  an 
adequate  performance  measurement  and  record  keeping  system 
within  training  systems  which  is  capable  of  collecting 
student  performance  data  of  fine  enough  detail  and  with 
sufficient  reliability  to  produce  dependable  conclusions, 
and  (2)  that  a  data  recording  system  is  in  place  on  a 
continuous  basis  as  training  devices  are  used  to  gather  time 
and  usage  data  to  be  used  in  conjunction  with  the 
performance  data  . 

Figure  2  contains  a  flowchart  representation  of  the  media 
selection  process.  The  description  which  follows  refers  to  the 
individual  steps  in  this  chart.  The  initial  input  to  the  media 
selection  process  is  a  complete  list  of  cri ter  ion-referenced 
objectives  (CROs)  and  instructional  objectives  from  objectives 
hierarcny  analysis.  Objectives  are  grouped  together  during 
syllabus-building  for  combined  mediation  as  instructional  "seg¬ 
ments".  For  the  F-16,  these  groups  did  not  average  over  three  to 
four  objectives  per  segment,  and  the  types  of  objective  (and  thus 
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tne  type  o£  instructional  strategy  required  to  teach  it)  were  not 
mixed  within  segments  wnere  it  could  be  avoided.  Though  tne  word 
"objective"  is  used  in  this  report  to  designate  the  instructional 
event  to  which  a  medium  will  be  assigned,  it  should  be  understood 
that  tne  eventual  assignment  will  be  an  instructional  segment. 
These  segments  make  up  tne  inventory  of  individual  instructional 
events  which  must  be  mediated.  The  derivation  of  CROs  and  a 
description  of  their  purpose  is  contained  in  F-16  Aircrew  Train¬ 
ing  Development  Project  Report  No.  5,  "Derivations,  Formatting, 
and  Use  of  Criterion-Referenced  Objectives  (CROs)  and  Criterion- 
Referenced  Tests  (CRTs)".  The  same  description  for  objectives 
hierarchy  analysis  is  contained  in  project  report  no.  8, 
"Objectives  Hierarchy  Analysis  Methodology  Report". 

A  final  note  on  bookkeeping  requirements  and  the  handling  of 
changes  is  in  order.  In  the  execution  of  many  ISD  processes, 
particularly  in  large-scale  ins tr uc tional  development  where  much 
information  and  data  are  generated,  the  bookkeeping  process 
required  becomes  extremely  important.  It  is  unavoidable  for  the 
developer  to  become  disoriented  in  the  midst  of  a  disorganized 
pile  of  data  which  he  generates,  but  with  an  adequate  bookkeeping 
process,  it  is  easy  to  make  large  numbers  of  media  decisions 
systematically.  It  is  also  easy,  given  appropriate  bookkeeping 
practices,  to  insert  changes  to  the  body  of  the  data  base  created 
by  media  selection.  On  the  other  hand,  without  such  systematic 
procedures,  especially  for  bookkeeping  on  decisions  made,  changes 
made  can  result  in  confusion,  duplication  of  effort,  loss  of 
data,  misplaced  training  requirements,  and  eventually  wasteful 
results.  The  media  selection  process  described  in  this  document 
depends  upon  a  bookkeeping  system  of  forms  and  tables  which 
collect  and  summarize  data  for  individual  objectives  during  the 
media  selection  process.  These  preserve  a  data  trail  for  review 
and  use  by  tnose  charged  with  revision  responsibilities.  There 
is  nothing  sacred  about  the  system  of  forms  beyond  the  fact  that 
they  represent  a  bookkeeping  system  wnich  is  adequate  to  tecord 
the  decisions  made  without  losing  them.  Alternate  metV'ds  of 
bookkeeping,  particularly  computer  i  za  tion  of  the  prc*  are 

acceptable  and  desirable  to  the  extent  that  they  minimize  the 
bookxeeping  effort  without  decreasing  the  developer's  ability  to 
execute  the  process  and  keep  from  losing  his  orientation  within 
the  vast  process  of  media  selection. 


3.1  Step  1:  Sort  Objectives  into  Academic  and  Training  Device 
Groups 

During  this  step,  CROs  and  objectives  were  sorted  into  two 
classes.  The  first  class  was  made  up  of  all  those  objectives 
whose  behavior  require  real  or  simulated  job  environments  or 
.equipment.  This  was  the  "training  device"  group  of  objectives. 
The  second  class  contained  objectives  whose  benaviors  can  be 
performed  without  such  equipment  or  environments.  This  was  the 
"academic"  group  of  objectives. 
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There  were  two  keys  to  making  this  discrimination.  The 
first  was  the  exact  wording  of  the  behavior.  If  the  behavior 
read  "perform  the  pre-takeoff  inspection,"  then  the  behavior 
required  equipment  to  be  inspected.  If  on  the  other  hand  the 
behavior  read  "list  the  items  to  be  inspected  in  the  pre-takeoff 
inspection"  no  equipment  was  indicated,  and  the  behavior  could  be 
accomplished  outside  of  a  real  or  simulated  job  setting. 

Conditions  and  standards  of  an  objective  or  CRO  were  the 
second  key  to  making  the  discrimination.  A  condition  may  specify 
the  type  of  display  to  be  presented  to  the  student.  For 
instance,  a  condition  could  require  the  student  to  identify 
certain  items  from  a  radar  return  "given  a  photograph  of  a 
return".  In  this  case  the  type  of  performance  environment  was 
specified.  In  the  absence  of  such  direction,  a  real-world 
environment  was  assumed. 

The  purpose  of  this  sorting  exercise  was  to  determine  whicn 
one  of  the  two  media  selection  matricies  each  objective  would  be 
processed  tnrough:  Academic  selection  matrix  or  training  device 
selection  matrix.  Worksheets  to  facilitate  the  bookkeeping 
chores  of  media  selection  were  set  up  at  this  time.  For  each 
academic  objective  the  appropriate  numerical  entry  was  made  in 
Table  1,  the  academic  objective  analysis  worksheet.  For  training 
device  objectives,  Table  2,  the  training  device  objectives 
analysis  worksheet,  was  filled  in.  Later  steps  added  data  to 
these  worksheets,  which  were  simply  set  up  during  this  step. 


3.2  Step  2:  State  all  Nonstrategy  Factors  Affecting  Media 
Selection 

The  nonstrategy  factors  affecting  media  selection  in  any 
given  situation  can  be  so  numerous  as  to  defy  a  complete  listing. 
There  are  certain  general  classes  of  constraints  which  need  to  be 
examined  across  all  situations,  since  they  can  have  serious 
effects  on  the  potential  successful  implementation  of  the  media 
if  not  taken  into  account.  The  most  important  of  the  constraints 
are  listed  below: 

a.  The  acquisition  cost  of  the  media  equipment. 

b.  The  maintenance  cost  of  the  media  equipnent. 

c.  The  characteristics  of  the  media  which  enhance  motiva¬ 
tion  and  maintain  attention  and  engagement  of  the 
student  with  the  instructional  task. 

d.  The  "standardization"  of  the  media  device  (ready  avail¬ 
ability,  vendor  support,  documentation,  etc.)  . 

e.  Environmental  safety  or  other  "special"  support  require¬ 
ments  . 
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g.  Suitability  for  environments  in  which  training  will  take 
place  . 

These  factors  were  utilized  later  in  the  procedure  in  making  the 
final  decisions  between  those  subset  members  of  the  entire 
instructional  media  pool  which  met  the  threshold  requirements  of 
the  initial  selection  process.  The  main  factors  influencing 
media  selection  for  F-16  have  been  presented  in  Section  2.3  and 
2.4  of  this  report. 


3.3  Step  3:  List  the  Media  Pools 

The  next  step  was  to  generate  the  media  pools  from  which 
instructional  media  would  be  selected.  This  required  the 
development  of  two  separate  lists:  Academic  media  and  training 
device  media.  These  inventories  were  developed  by  joint  OTD 
team-contractor  effort.  In  order  to  ensure  a  selection  that 
would  provide  maximum  flexibility,  these  pools  consist  of  all 
instructional  media  within  existing  Air  Force  inventories  as  well 
as  those  projected  to  be  available  during  the  next  decade.  Tn i s 
allowed  the  OTD  team  to  produce  a  media  selection  for  the  opening 
RTU  class  in  March  1980,  as  well  as  the  blueprint  media  selection 
for  the  future.  This  is  discussed  in  Step  5,  reconfigure  media 
to  produce  alternative  media  lists. 


ACADEMIC  MEDIA  POOL 


1.  Computer-assisted  instruction  plus  lesson  guide:  An 

ind  iv id  ual i zed  d  ig i tal  compute r-ba  sed  Iristr uctional  system 
whereby  direct  student-computer  interface  allows  the  auto¬ 
matic  management  and  display  of  information  to  the  student, 
the  acceptance,  processing  and  evaluation  of  student 
responses,  the  display  of  results,  and  the  selection  of 
subsequent  learning  events  plus  a  worksheet  containing  tne 
lesson  (segment)  identification  number  and  title,  an  intro¬ 
duction,  list  of  objectives,  support  information,  graphics, 
practice  items,  and  feedback  to  provide  the  student  with  a 
written  record  of  all  mediated  presentations. 

2.  Computer-assisted  instruction  (CAI)  plus  videotape  plus 
lesson  guide:  CAI  plus  lesson  guide  pTus- coTor  tel ev  I s ion 
program  stored  on  a  tape  cassett  which  may  be  operated  by 
the  student. 


3.  Computer-assisted  instruction  plus  videotape  plus  part-task 
traiher’Dlus  lesson  quidel  ^A I  plus  videotape  Dlus  lesson 


practice  of  a  single  or  limited  range  of  tasks  whicn  are 
cued  and  prompted  by  a  videotape  display. 


4.  Interactive  part-task  trainer  plus  1  essonju ide  :  Lesson 

g  ulJe  pTus  part- task  trainer  ETiaT  is  wired  to  a  computer  to 
allow  realistic  responses  to  student  manipulations.  Cockpit 
gauges,  for  example,  respond  to  student  pressure  on  the 
throttle . 

5.  Random  access  slide  plus  lesson  guide:  lesson  guide  plus 
visual  slTde  proj  ec  tion  "system  that  allow  slides  to  be 
presented  in  any  order,  rather  than  following  the  order  in 
which  they  are  put  into  a  cartridge. 

6.  Motion  picture  plus  lesson  guide;  Lesson  guide  plus  16mm  or 
supe  r  3  mm  coTor  f  ITm  project  ion  system. 

7.  Videotape  plus  lesson  guide;  See  2  above. 

8.  videodisc  plus  lesson  guide:  Lesson  guide  plus  color 
talev  ision  prog  ram  stored  on  a  thin  plastic  disc  instead  of 
a  tape  (as  in  2  above) ,  allowing  random  access,  freeze 
frame,  and  motion. 

9.  Tape/Slide  plus  learning  guide:  Learning  guide  plus  system 
for  presenting  audiovisual  information  by  means  of  an  audio 
tape  and  a  series  of  synchronized  projected  visual  slides. 

1 0 .  Suitcase  projector  with  audio  and  both  still  and  motion 
visual  plus  lesson  guide:  Lesson  guide  plus  portable 
desk- top  screen  For  viewing  audiovisual  film  strip  loops 
that  include  both  still  slides  and  limited  motion  sequences. 

11.  Workbook  plus  ' ^sson  guide:  Lesson  guide  plus  printed 
booklet  which  Includes  lesson  identification  number  and 
title,  introduction,  objectives,  core  idea  information, 
discussion  of  core  idea  information,  graphics,  practice,  and 
f eedbac  k  . 

12.  Color  workbook  plus  lesson  guide:  Lesson  guide  plus 
workbook  as  in  11  above,  but  with  color  graphics. 

13.  Workbook  plus  slides  plus  lesson  guide:  See  5  and  11  above. 

14.  Workbook  plus  audio  plus  lesson  guide:  See  11  above  plus 
aud  lo  tape  recording  that  may  be  reviewed  by  individual 
student  with  tape/player/or  presented  to  group  as  in  21,  26 
and  30  below. 

15.  Programmed  text  plus  lesson  guide:  Lesson  guide  plus  a 
pr  inted  text  conta in ing  a  sequence  of  small  frames  of 
information  and  required  a  simple  written  response  from  the 
student . 
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16.  Training  manual  plus  lesson  guide:  Lesson  guide  plus  off 
FEe  sfieTT  tecnn Teal  pubi  ication  such  as  F-16  Dash  One, 
operator  guide  or  maintenance  manual. 

17.  Model/Actual  equipment  plus  lesson  guide:  Lesson  guide  plus 
scale  model  or  actual  item  o£  operational  equipment  used  for 
training  purposes. 

1 8 .  Cockpit  familiarization  trainer  ( CFT)  plus  lesson  guide : 
Lesson  g  u  ide  plus  F-16  3-d imens  lonal  life-size  cockpit 
mockup  with  screen  for  tape/ si  ides  located  where  head-up 
display  wo  uld  go  . 

19.  CFT  plus  tape/slide  plus  lesson  guide:  See  9  and  18. 

20.  Lecture  plus  lesson  guide:  Lesson  guide  plus  formal, 
stand-up  podium  expository  presentation  to  a  large  class  (10 
or  mor  e)  . 

21.  Lecture  plus  audio  and  lesson  guide:  See  14  and  20. 

22.  Lecture  plus  visual  motion  plus  lesson  guide:  See  20  plus 
2-v  ideo  tape  or  6-motion  picture  above. 

2  3 .  Lecture  plus  model/actual  equipment  plus  lesson  guide :  Se e 
17  and  20  above.  "  ‘ 

24.  Lecture  plus  student  response  system  plus  lesson  guide:  See 
20  above  plus  desk-mounted  responders  that  allow  all 
students  to  respond  to  instructor  questions  simultaneously 
and  consequently  guide  lecture  delivery  based  on  real-time 
student  comprehension. 

25.  Tutorial  plus  lesson  guide:  Lesson  guide  plus  one-on-one 
student-instructor  interaction  loosely  structured  around  a 
flexible  format  of  objectives. 

26.  Tutorial  plus  audio  plus  lesson  guide:  See  21  and  25  above. 

27.  Tutorial  and  visual  motion  plus  lesson  guide:  See  22  arid 

JT. - 

28.  Tutorial  plus  model/actual  equipment  plus  lesson  guide:  See 
23  and  25  above  . 

29.  Seminar  plus  lesson  guide:  Lesson  guide  plus  informal 
d  i  sc  uss  ion  loosely  structured  around  a  firm  format  of 
objectives  for  a  small  class  (2  to  9  students)  . 


See  21  and  29. 


2  3  and  2  9. 


TRAINING  DEVICE  MEDIA  POOL 


1.  panel  mockup 

2.  Cockpit  mockup 

3.  Stick  and  throttle  trainer 

4.  Stores  management  system  trainer 

5.  Avionics  display 

6.  2-dimensional  isolated  system  device 

7.  Radar  warning  receiver  trainer 

8.  Interactive  CFT 

9.  CFT 

10.  Egress  Procedures  Trainer  (EPT) 

11.  Dynamic  system  simulator 

12.  Simulation  air-to-air  combat  trainer 

13.  Operational  Flight  Trainer  (OFT) 

14.  OFT  and  night  visual  system  trainer 

15.  OFT  and  Digital  Radar  Land  Mass  System 

16.  OFT  and  electronic  warfare  trainer 

17.  Weapons  System  Trainer  (WST) 

18.  F-16A  aircraft 

19.  F-16B  aircraft 

20.  Advanced  Simulator  for  Pilot  Training  (ASPT) 

3.4  Step  4:  Evaluate  the  Instructional  Characteristics  of  All 
Media 

Once  the  two  media  pools  have  been  identified,  each  must  be 
evaluated  in  terms  of  its  training  capabilities.  This  process 
was  completed  by  developing  a  media-by-capabilities  matrix.  The 
media  in  each  media  pool  were  listed  across  the  top  of  the 
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matrix,  while  an  inventory  of  relevant  training  capabilities  was 
listed  down  the  side.  This  yielded  a  box,  or  cell,  for  each 
medium  for  each  capability.  These  cells  were  then  filled  with 
numbers  representing  a  1-5  rating.  If  the  medium  did  not  possess 
or  exhibit  the  capability  the  cell  was  left  blank.  If  the  medium 
exhibited  the  capability  to  a  very  limited  degree,  it  was  given  a 
rating  of  1.  If,  on  the  other  hand,  it  exhibited  that  capability 
to  a  great  degree,  it  was  rated  4  or  5.  The  scale  was  as 
follows : 

5--IDEAL:  This  is  the  best  medium  for  the  objective.  Its 

efficiency,  effectiveness,  and  flexibility  are 
outstanding.  The  scope  is  adequate  for  the  full  range  of 
possible  instructional  instances  and  there  is  a 
likelihood  of  positive  affect. 

4--DESIRABLE :  This  is  a  good  medium  for  the  objective.  It 

is  efficient,  effective,  and  flexible.  A  broad  range  of 
possible  instructional  instances  are  available,  and  there 
is  potential  for  positive  affect. 

3--G00D:  This  medium  will  teach  the  objective  adequately. 

It  is  reasonably  effective,  efficient,  and  flexible.  It 
will  provide  the  most  commonly  required  instructional 
i nstances  . 

2 — ADEQUATE:  This  medium  is  acceptable,  but  weak.  The 

scope  and  flexibility  are  limited  and  both  effectiveness 
and  efficiency  are  poor.  There  may  be  a  strong 
dependence  on  secondary  media. 

1  —  POOR:  This  is  the  medium  of  last  resort.  Effectiveness, 
efficiency,  scope  and  flexibility  are  poor.  There  may  be 
almost  complete  reliance  on  secondary  media  and  there  is 
a  potential  for  negative  effect. 

The  academic  and  training  device  media  were  analyzed  on  the 
basis  of  two  very  different  sets  of  training  capabilities.  The 
capabilities  rated  for  the  academic  and  training  device  media 
were  as  follows: 

3.4.1  Display  Characteristics 

Those  features  which  are  an  integral  part  of  the  media 
output . 

1.  Audio-voice:  An  output  of  both  human  as  well  as  nonhuman 

audible  signals. 

2.  Audio:  An  output  of  nonhuman  audible  signals  (i.e.,  music, 

tones ,  etc . ) . 

3.  Tactile:  A  display  which  feels  the  same  to  the  touch  as 

real  world  equipment  (e.g.,  knob  on  the  display  of  a  cockpi  t 
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training  panel  feels  the  same  to  the  touch  as  the  same  knob 
on  the  actual  equipment)  . 


4.  Kinesthetic:  Equipment  on  the  display  responds  the  same  way 
to  the  touch  as  real  world  equipment  (e.g.,  a  switch  on  the 
display  responds  exactly  like  the  switch  on  the  aircraft)  . 

5.  Visual:  The  following  display  charac ter i st ics  are  concerned 
with  the  visual  display  seen  by  the  student. 

a.  Text:  The  display  includes  printed  material. 

b.  Pictorial:  The  display  includes  photographs. 

c.  Diagrams/Drawings:  The  display  includes  line  art  or 
full  art  drawings. 

d.  Spatial  Layout:  The  display  has  the  exact  same 
dimensions/proportions  as  the  real  equipment  it  depicts. 

e.  Motion:  The  display  includes  movement. 

f.  Color:  The  display  includes  color. 

3.4.2  Response 

The  type  of  student  input  accepted  by  the  medium. 

1.  Verbal/wr i tten :  The  student  response  may  be  either  orally 
given  or  handwritten. 

2.  PTM--The  student  may  respond  by  pointing  (P),  touching  (T), 
or  marking  ( M)  an  item. 

3.  Manipulate:  The  student  may  respond  by  operating  or 
controlling  a  device  or  piece  of  equipment. 

4.  Covert:  Tne  student's  response  is  undetectable  (i.e.,  ne 
conceives  the  answer  but  does  not  make  an  observable 
response.  (NOTE:  This  category  of  response  has  proved  to 
be  generally  impractical,  in  that  there  is  no  method  of 
detecting  whether  the  student's  answer  is  correct  or 
incorrect.  Therefore,  it  has  not  been  used  in  developing 
the  F-16  training  system. 

3. 4. 3  Evaluation 

The  method  of  delivering  a  response  to  the  student  on  the 
correctness  or  incorrectness  of  his  answer. 

1.  instructor:  A  response  to  the  student's  answer  is  given  by 
the  instr  uc  tor . 

2.  Automated:  Tne  response  to  the  student's  answer  is 
generated  by  the  medium. 


-Q _ m  •  ->■- 
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Peer:  The  response  to  the  student's  answer  is  given  by  a 

fellow  student.  (NOTE:  This  is  generally  not  acceptable  as 
key . 

3.4.4  Feedback 

The  type  of  response  the  student  receives  after  answering  a 
test  or  practice  item. 

1.  Frequency:  How  often  feedback  is  presented  to  the  student. 

a.  immed  iate/ Response  :  The  student  receives  feedback 
immediately  after  answering  a  test/ prac t ice  item. 

b.  Immed  iate/ Er  ro  r  :  The  student  receives  feedback 
immediately  after  answering  a  test/practice  item 
incor rec  tly  . 

c.  Periodic:  The  student  receives  feedback  after  answering 
a  predetermined  number  of  tes  t/ prac  t  ice  items. 

d.  Post-Session:  The  student  receives  feedback  after  a 
complete  training  session. 

2.  Content:  The  information  provided  to  the  student  during 
feedback  . 

a.  Correct  Answer:  The  student  is  given  the  right  answer 
after  responding  to  a  question  incorrectly. 

b.  Right/Wrong  Elaborated:  The  student  is  given  informa¬ 
tion  as  to  why  he  answered  a  question  correctly  or 
incorrectly. 

c.  Branch  to  New  Display:  The  student  receives  new 
information  based  on  the  correctness  or  uncorrectness  of 
his  answers  ( e  .g  .  ,  If  the  student  answers  correctly  the 
display  may  branch  to  a  new  lesson.  If  the  student 
answers  incorrectly  the  display  may  branch  to  additional 
practice  items  or  a  "HELP"  segment)  . 

3.4.5  Special  Requirements 

Unusual  needs  required  by  a  specific  lesson  which  must  be 
performed  by  the  selected  medium. 

1.  Crew/Team  Interaction:  The  need  for  the  media  to  allow 
interaction  required  between  crew  members  and  the  flight 
team  (e.g.,  interaction  between  a  pilot  and  a  navigator)  . 

2.  Environment:  The  need  for  the  media  to  present 
extraordinary  conditions  which  the  student  needs  to  respond 
to  (e.g.,  taking  off  during  unfavorable  weather)  . 
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Motion:  The  need  for  the  media  to  present  normal  and 
abnormal  movement  which  require  a  specific  response  from  the 
student  (e.g.,  the  student  may  need  to  make  certain 
responses  wnile  an  aircraft  is  banking) . 

Time  Variability:  The  need  for  the  media  to  present 
information  in  fast  or  slow  motion. 

Learner  control:  The  need  for  the  student  to  regulate  his 
own  learning  by  controlling  content,  sequence,  and  rate. 


TRAINING  CAPABILITIES  OF  ACADEMIC  MEDIA 

Display  characteristics:  Those  features  wnich  are  an 

integral  part  of  the  media  output. 

A.  Audio-voice:  An  output  of  both  human  as  well  as 
nonhuman  audible  signals. 

B.  Audio:  An  output  of  nonhuman  audible  signals  (i.e., 
music,  tones,  etc.)  . 

C.  Tactile:  A  display  wnich  feels  the  same  to  the  touch  as 
real  world  equipnent  (e.g.,  knob  on  the  display  of  a 
cockpit  training  panel  feels  the  same  to  the  touch  as 
the  same  knob  on  the  actual  equipment)  . 

D  Kinesthetic:  Equipment  on  the  display  responds  the  same 
way  to  tne  touch  as  real  world  equipment  (e.g.,  a  switch 
on  the  display  responds  exactly  like  the  switch  on  the 
a  i  rcra  f  t)  . 

E.  Learner  control:  The  need  for  the  student  to  regulate 
his  own  learning  by  controlling  content,  sequence,  and 
r  a  te  . 


TRAINING  CAPABILITIES  OF  ACADEMIC  MEDIA 

Physical  layout  characteristics:  The  three  categories 
within  this  class  define  the  degree  of  realism  with  wnich 
the  cockpit  or  other  training  environment  is  laid  out. 

A.  Conformity  with  shape:  A  high  rating  on  this  character¬ 
istic  indicates  a  training  device  ir  which  the  actual 
flight  equipment  is  simulated;  e.g.,  round  dial  wi  th  a 
needle  is  shown  as  a  round  dial  and  not  a  digital 
readout . 

B.  Be  tween-panel  spatial  relations:  A  high  rating  on  this 
characteristic  indicates  a  training  device  where  the 
absolute  distance  between  major  panels  and  subpanels  is 
a  close  approx ima t ion  of  the  real  equipment. 


29 


C.  Wi thin- panel  spatial  relations:  A  high  rating  on  this 
characteristic  indicates  a  training  device  in  which  the 
absolute  distance  between  items  on  an  individual  panel 
is  a  close  approximation  of  the  real  equipnent. 

Stimulus  pro pe r ties- - in te r io r :  These  characteristics  define 
the  fidelity  with  which  the  control  indicators  and  displays 
located  within  the  cockpit  present  information  to  the 
student  . 

A.  Fidelity  of  visual  stimulus:  This  characteristic 
defines  the  fidelity  of  indicators  and  displays  which 
present  information  by  visual  means. 

B.  Fidelity  of  audio  (nonvoice)  stimuli:  This  character¬ 
istic  defines  the  fidelity  with  which  tne  trainer 
presents  nonvoice  audio  stimuli  such  as  the  AIM-9 
Missile  Aural  Tone. 

C.  Fidelity  of  kinesthetic  stimulus:  This  characteristic 
defines  the  degree  to  which  the  "feel"  of  operating  the 
trainer's  interior  controls  correspond  to  the  "feel"  of 
operating  the  controls  of  the  actual  equipnent. 

D.  Voice  stimulus  capability:  The  presence  of  this 
character istic  indicates  the  student  can  receive  spoken 
communication  from  the  instructor. 

Stimulus  properties — exterior:  This  class  defines  the 
fidelity  with  which  the  environment  external  to  the  actual 
aircraft  is  replicated  in  the  training  device. 

A.  Field  of  view:  A  high  rating  on  this  characteristic 
indicates  a  field  of  view  which  is  large  enough  to 
approximate  the  field  of  view  of  the  student  in  the 
actual  equipnent. 

B.  Proportion  of  distant  objects:  A  high  rating  on  this 
characteristic  indicates  a  visual  system  with  enough 
resolution  so  that  objects  at  a  great  apparent  distance 
appear  to  be  in  their  true  proportions  and  are  of  the 
correct  apparent  size. 

C.  Coloration  and  shad ing--obj ects :  A  high  rating  on  this 
character istic  indicates  realistic  color  and  shading  so 
that  apparent  depth  is  enhanced. 

D.  Visual  sensation — motion:  A  high  rating  on  this 

c  na  r ac te r ist  ic  indicates  a  fully  dynamic  display  in 
which  apparent  motion  is  of  high  fidelity.  A  lower 
rating  on  this  would  indicate  either  stop  action  motion 
or  still  visuals. 


E.  Shape  of  objects:  A  high  rating  on  this  characteristic 
indicates  no  matter  what  the  apparent  aspect  angle  of 
the  visual  display,  the  object  display  will  be  of  the 
correct  proportion. 

F.  Resol ution--detai 1  given  distance:  A  high  rating  on 
this  cha ra : te r i st i c  indicates  that  the  relationship 
between  the  amount  of  detail  with  which  an  object  is 
displayed  and  the  apparent  distance  the  student  is  from 
that  object  approximates  the  amount  of  detail  that  would 
be  visible  from  the  actual  equipment  in  that  same 

s  i  tua  t i on  . 

Manipulative  properties:  The  manipulative  properties  of  a 
training  device  define  the  ways  and  the  fidelity  with  which 
a  student  can  interact  with  the  controls  of  the  trainer. 

A.  Fidelity  of  manipulation:  A  high  rating  on  this 
characteristic  indicates  trainer  control  switches  that 
closely  approximate  the  controls  of  the  actual  equipment 
such  that  they  are  manipulated  in  the  same  manner. 

B.  Timely  manipulation--responds  connection:  A  high  rating 
on  this  characteristic  indicates  that  the  training 
device  responds  to  student  with  a  time  lag  which  closely 
approximates  the  time  lag  present  in  actual  equipment. 

Response  properties:  This  class  defines  the  realism  with 
which  the  training  device  responds  to  student  actions. 

A.  Fidelity  of  interior  response:  This  characteristic 
defines  the  fidelity  with  which  the  training  device 
responds  to  student  actions.  It  differs  from  class  two, 
above  in  that  it  refers  to  changes  in  the  interior  state 
of  the  training  device  which  are  student  generated  as 
opposed  to  device  generated. 

B.  Fidelity  of  exterior  response:  This  characteristic 
refers  to  the  fidelity  with  which  changes  in  the 
external  environment  based  on  student  actions  are 
represented  . 

Instructional  features:  The  list  of  characteristics  which 
comprise  this  class  is  not  an  exhaustive  one;  rather,  it  is 
a  catalog  of  special  instructional  features  which  required 
significant  hardware  engineering,  but  which  are  necessary 
for  instruction  in  a  reasonable  large  subset  of  in-flight 
tests  . 

A.  Stop  action  capability:  The  presence  of  this  character¬ 
istic  gives  the  instructor  the  ability  to  halt  the  flow 
of  real  time  at  any  point  in  the  training  device 
sessions  so  that  the  student  may  study  the  displays  and 
situation  at  that  critical  instant. 


B.  Canned  demo  capability:  The  presence  of  this  character¬ 
istic  means  that  the  training  device  possesses  the 
capability  of  presenting  stored  scenarios.  The  higher 
the  rating  on  this  characteristic,  the  greater  the 
degree  of  self-stimulation  of  displays  and  controls  that 
is  possessed  by  the  training  device. 

C.  Live  demo  capability:  The  presence  of  this  character¬ 
istic  indicates  that  the  training  device  is  capable  of 
being  manipulated  externally  by  the  instructors  such 
that  the  student  can  see  the  sequence  of  events  and 
actions  in  any  scenario  that  the  instructor  can  imagine. 
A  high  rating  on  this  characteristic  can  be  likened  to 
the  training  possibilities  that  exist  in  a  dual  control 
aircraft. 

D.  Conditions  reset  capability:  The  presence  of  this 
characteristic  indicates  that  the  training  device  has 
the  capacity  to  store  the  particular  set  of  parameters 
such  that  upon  the  command  of  the  instructor,  the  state 
of  all  displays  and  controls  can  be  returned  to  that 
stored  state  of  conditions.  The  high  ratings  on  this 
character istic  indicate  either  larger  numbers  of  initial 
conditions  which  may  be  stored  and/or  larger  numbers  of 
parameters  which  are  stored  within  any  one  set  of 

cond i tions . 

7.  Evaluation  features:  This  class  is  similar  to  the  preceding 
one  in  that  the  list  of  characteristics  do  not  exhaust  the 
class,  rather  they  are  characteristics  whose  presence  are 
often  desired/required  due  to  the  increase  in  training 
efficiency  they  engender  by  providing  the  immediate  and 
sufficient  feedback  to  the  student  and/or  the  performance 
evaluation  capabilities  they  provide  which  would  be  lacking 
without  this  characteristic. 

A.  Instructor  view  of  student  displays:  Ratings  on  this 
characteristic  define  the  extent  with  which  the  displays 
available  to  the  student  are  duplicated  for  the  benefit 
of  the  instructor  to  use  in  monitoring  student 

per formance  . 

B.  R/W  feedback:  The  presence  of  this  cha rac ter i s t i c 
indicates  that  the  training  device  has  a  capability  of 
automatically  monitoring  student  performance  and 
indicating  to  the  student  whether  that  performance  is 
acceptable  or  unacceptable.  No  elaboration  is  provided; 
merely  indication  of  behavior. 

C.  Out  of  tolerance  feedback:  The  presence  of  this 
characteristic  indicates  that  the  training  device  has 
the  capability  of  automatically  monitoring  student 
performance  and  indicating  to  him  when  that  performance 
does  not  meet  with  a  preset  standard  as  well.  High 
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ratings  on  this  characteristics  indicate  the  increased 
accuracy  with  which  the  training  device  can  present  the 
student  with  the  degree  to  which  he  has  failed  to  meet  a 
given  standard. 

D.  Instructor  feedback:  The  presence  of  this  capability 
indicates  that  the  instructor  is  present  with  a  student 
and  can  monitor  his  performance  such  that  the  human 
judgement  factor  is  included  in  the  feedback  equation. 

E.  Replay  of  student  performance:  The  presence  of  this 
characteristic  indicates  that  the  training  device  has 
the  capability  of  storing  a  record  of  the  actions  taken 
by  the  student  such  that  on  command  it  can  "replay" 
those  actions  in  terms  of  a  real  time  duplication  of 
displays  and  control  movements  so  that  the  student  may 
reveal  an  exact  record  of  his  own  performance. 

8.  Opponent:  This  class  defines  the  amount  of  combat  training 
possible  with  a  device.  It's  characteristic  is  to  define 
the  type  and  capabilities  of  opponents  which  the  student  can 
meet  in  the  training  device. 

A.  Maneuvering  opponent:  The  presence  of  this  character¬ 
istic  indicates  that  the  training  device  is  able  to 
present  the  student  with  an  opponent  who  will  maneuver 
in  response  to  the  student's  own  control  inputs.  The 
higher  the  rating  in  this  characteristic,  the  higher  the 
fidelity  with  which  the  maneuvering  opponent  is 
simulated . 

Table  3  presents  the  media  selection  matrix  for  academic 
media,  while  Table  4  presents  the  matrix  for  training 
device  media.  Again,  an  empty  cell  indicates  that  the 
medium  does  not  possess  the  training  capability  at  all, 
a  1  suggests  that  the  medium  possesses  it  to  a  very 
limited  degree,  and  a  5  indicates  that  the  medium 
excells  in  that  capability. 


3.5  Step  5:  Reconfigure  Media  to  Produce  Alternate  Media  Lists 

Once  the  strategy  and  nonstrategy  factors  have  been  stated 
which  will  limit  or  constrain  media  selection,  the  master  list  of 
all  available  media  can  be  scanned  for  media  which  do  not  support 
the  factors,  or  which  are  outside  the  bounds  of  the  constraints. 
These  media  are  then  designated  on  the  candidate  media  list. 

Some  possible  reasons  for  designating  a  medium  might  be: 

-  The  immediate  or  long-range  costs  associated  with  a  medium 
for  production,  repair,  acquisition,  or  operation  are 
prohibitively  high  when  weighed  against  the  expected 
benefits  . 
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-  Tne  expertise  or  equipment  required  to  produce  and  modify 
instructional  presentations  are  not  readily  available. 

-  The  medium  does  not  fall  within  the  limits  set  by  the 
constra ints . 

-  The  medium  is  as  yet  experimental  in  nature  and  has 
unknown  operational  properties  (durability,  student 
acceptance,  etc)  . 

-  The  production  time  for  presentations  in  this  medium  is 
too  long  . 

-  Tne  medium  is  not  durable  and  is  subject  to  breakdowns. 

-  The  medium  makes  demands  of  students  that  are  beyond  their 
normal  range  of  interest  or  ability. 

-  The  medium  is  too  complex  to  use  and  is  likely  to  require 
too  much  of  the  student's  time  or  attention  for  operation. 

These  are  suggested  media  designation  criteria  and  do  not 
comprise  an  exhaustive  list.  A  complete  list  for  a  given  media 
selection  will  depend  upon  factors  local  to  the  project.  In 
making  designations  some  judgement  is  required.  Decisions  are 
not  likely  to  be  clear-cut,  and  some  trade-offs  may  be  considered 
before  designating  media.  For  instance,  some  media  may  be 
clearly  unaf fordable ,  too  delicate,  not  portable,  or  in  some 
other  way  beyond  practicality.  Other  media,  on  the  other  hand, 
may  appear  to  be  out  of  bounds  on  some  factors  but  yet  have  such 
a  strong  positive  recommendation  from  others  that  they  may  be 
retained,  unlabeled  on  the  list.  A  medium  may  be  retained  which 
is  expensive  in  acquisition  but  yet  can  implement  instructional 
strategies  far  better  than  other  media.  A  medium  which  has  found 
itself  in  this  situation  in  the  past  is  computer-assisted 
instruction  (CAI) .  The  initial  cost  of  a  CA I  system  is  often 
troublesome,  but  the  attraction  of  the  strategy  versatility  of  a 
computer  and  the  side  uses  of  a  computer,  for  instance,  in 
computer-managed  instruction  (CMI),  maxa  a  CAI  system  very  desir¬ 
able  despite  cost.  (Surprisingly,  when  total  actual  costs  are 
compared  for  various  media,  some  CAI  systems  turn  out  to  be 
comparatively  less  expensive.  This  is  because  actual  costs  are 
in  fact  higher  than  usually  estimated  and  CAI  costs  lower  than 
expected.)  This  combination  could  cause  a  medium  to  be  left  on 
the  candidate  list  undesignated,  despite  some  factors  legislating 
against  it.  This  has  been  the  case  for  F-16.  When  media  are  for 
some  reason  desirable  but  cannot  fit  the  requirements  imposed  by 
the  factors,  reconfiguration  of  a  medium  by  combination  with 
another  medium  or  by  different  patterns  of  use  should  be  consid¬ 
ered.  Two  media  used  in  combination  may  be  more  effective  than  a 
single  med  ium . 

The  goal  of  this  reconfiguration  was  to  produce  two  media 
selections,  one  for  the  March  1980  syllabus  and  one  for  a 
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futuristic  unconstrained  syllabus.  project  personnel  referred  to 
the  1980  media  selection  as  the  "real  world"  selection  and  to  the 
futuristic  selection  as  the  "2001"  selection.  The  F-16  OTD  team 
reviewed  the  media  pools  for  academic  and  training  device 
instructional  media,  designating  with  an  asterisk  (*)  those  which 
would  not  be  available  for  the  real  world  syllabus  in  1980,  but 
would  probably  be  available  for  the  2001  syllabus.  These 
designations  are  listed  below: 

ACADEMIC  MEDIA  POOL  WITH  MEDIA  UNAVAILABLE  FOR  1980 
IMPLEMENTATION  DESIGNATED  WITH  AN  ASTERISK  (*) 

1. *  CAI  plus  lesson  guide 

2. *  CAI  plus  videotape  plus  lesson  guide 

3. *  CAI  plus  videotape  plus  part-task  trainer  plus  lesson  guide 

4. *  Interactive  part-task  trainer  plus  lesson  guide 

5. *  Random  access  slide  plus  lesson  guide 

6.  Motion  picture  plus  lesson  guide 

7.  videotape  plus  lesson  guide 

8. *  Videodisc  plus  lesson  guide 

9.  Tape/Slide  plus  lesson  guide 

10. *  Suitcase  projector  with  audio  and  both  still  and  motion 

visual  plus  lesson  guide 

11.  Workbook  plus  lesson  guide 

12. *  Color  workbook  plus  lesson  guide 

13.  Workbook  plus  slides  plus  lesson  guide 

14.  Workbook  plus  audio  plus  lesson  guide 

15.  Programmed  text  plus  lesson  guide 

16.  Training  manual  plus  lesson  guide 

17.  Model/Actual  equipment  plus  lesson  guide 

18.  Cockpit  Familiarization  Trainer  (CFT)  plus  lesson  guide 

19.  CFT  plus  tape/slide  plus  lesson  guide 

20.  Lecture  plus  lesson  juide 
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Lecture  plus  audio  plus  lesson  guide 

22.  Lecture  plus  visual  motion  plus  lesson  guide 

23.  Lecture  plus  model/actual  equipment  plus  lesson  guide 

24.  Lecture  plus  student  response  system  plus  lesson  guide 

25.  Tutorial  plus  lesson  guide 

26.  Tutorial  plus  audio  plus  lesson  guide 

27.  Tutorial  and  visual  motion  plus  lesson  guide 

28.  Tutorial  plus  model/actual  equipment  plus  lesson  guide 

29.  Seminar  plus  lesson  guide 

30.  Seminar  plus  audio  plus  lesson  guide 

31.  Seminar  plus  visual  motion  and  lesson  guide 

32.  Seminar  plus  model/actual  equipment  plus  lesson  guide 

TRAINING  DEVICE  MEDIA  POOL  WITH  MEDIA  UNAVAILABLE  FOR  1980 
IMPLEMENTATION  DESIGNATED  WITH  AN  ASTERISK  (*) 

1.  Panel  mockup 

2.  Cockpit  mockup 

3.  Stick  and  throttle  trainer 

4.  Stores  management  system  trainer 

5.  Avionics  display 

6.  2-d imensional  isolated  system  device 

7.  Radar  warning  receiver  trainer 

8. *  Interactive  CFT 

9.  CFT 

10.  Egress  procedures  Trainer  (EPT) 

11.  Dynamic  system  simulator 

12.  Simulation  air-to-air  combat  trainer 
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13. *  Operational  Flight  Trainer  (OFT) 

14. *  OFT  and  night  visual  system  trainer 

15. *  OFT  and  Digital  Radar  Land  Mass  System 

16. *  OFT  and  electronic  warfare  trainer 

17. *  Weapons  System  Trainer  (WST) 

18.  F-16A  aircraft 

19.  F-16B  aircraft 

20.  Advanced  Simulator  for  Pilot  Training  (ASPT) 


3.6  Step  6:  Determine  Instructional  Media  Requirements  for  Each 
Objective 

This  step  required  the  media  selection  and  sequencing  staff 
to  go  through  the  complete  objectives  hierarchies  for  both  the 
academic  and  hands-on  syllabus  and  to  make  a  series  of  specific 
instr uctional  determinations  as  to  the  instructional  display 
requirements,  response  requirements,  evaluation  requirements, 
feedback  requirements,  and  other  special  requirements  imposed  t>y 
each  individual  instructional  objective  in  terms  of  both  the 
specific  content  characteristics  to  be  mastered  (visual  vs  verbal 
concept,  etc.)  and  the  instructional  strategy  implied  by  the 
instructional  classification  of  the  objective.  This  data  was 
collected  utilizing  structured  data  collection  instruments, 
designed  by  the  media  selection  team.  Table  5  represents  a 
partially  completed  academic  objective  analysis  worksheet,  while 
Table  5  depicts  a  partially  completed  training  device,  or 
hands-on,  analysis  worksheet. 

The  left  column  of  the  worksheet  represents  the  unique 
identification  number  associated  with  each  objective.  This 
allows  each  objective  to  be  designated  without  writing  it  out. 

The  check  marks  to  the  right  of  each  objective  number  indicate 
the  minimal  instructional  requirements  of  that  particular  objec¬ 
tive.  If  color  or  motion  are  absolutely  required  to  teach  the 
objective,  for  example,  they  receive  a  check.  If  they  are 
unnecessary,  they  do  not  receive  a  check.  The  pattern  of  check¬ 
marks  associated  with  an  objective  indicates,  in  the  opinion  of 
the  contractor  and  the  OTD  team,  the  minimal  requirements  an 
instructional  medium  must  provide  in  order  to  teach  that 
obj  ec  t ive . 

Some  general  guidelines  may  be  provided  for  specifying  the 
requirements.  For  display  capability  choose  the  least  compl i- 
cated  or  expensive  level  of  effort  which  will  accomplish  the 
instruction.  Un for t una tel y  there  is  some  subjectivity  in  this 
decision.  The  instruction  to  choose  the  least  complicated 
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capability  arises  from  the  tendency  of  media  research  to  find 
little  or  no  difference  in  learning  attributable  to  more  complex 
or  finely  finished  instructional  displays  (Travers,  1970). 

Simple  line  drawings  are  in  many  cases  as  i nst r uc t i ona 1 1 y  effec¬ 
tive  as  complex  renditions  or  even  photographs.  There  is  even 
some  evidence  that  during  training  the  more  complex  the  display, 
the  more  difficult  it  is  for  the  student  to  detect  the  m=sin 
message  or  salient  points. 

For  pace  and  control  decisions,  student  control  should  be 
favored,  depending  upon  the  anticipated  motivation  and  self- 
directedness  of  the  students.  Research  in  military  training 
environments  has  found  time  efficiencies  attendant  to  students 
being  given  control  of  instructional  order  and  speed  (Walker, 
1978) .  For  students  who  are  highly  motivated  and  sel f-directed 
this  effort  would  be  expected  to  be  very  strong.  On  the  other 
hand,  for  dealing  with  low-motivation  students  or  students  who 
are  not  self-directed,  student  control  may  be  inappropriate. 

The  logic  of  the  media  selection  process  should  now  become 
apparent.  First,  the  capabilities  of  the  media  are  determined. 
Second,  the  requirements  of  the  objectives  are  determined. 

Third,  the  computer  matches  the  capability  of  each  medium  to  the 
requirements  of  each  objective,  yielding  for  each  objective  a 
prioritized  list  of  those  media  most  ideally  suited  for 
presenting  each  objective. 

Once  data  were  collected  on  all  instructional  objectives, 
both  trainer  and  academic,  they  were  coded  and  entered  into  a 
high-speed  computer.  It  should  be  noted  that  the  collection  of 
over  25,000  data  points  was  an  extremely  time  consuming  task,  and 
partially  offset  the  advantage  gained  by  utilizing  the  data 
processing  capability.  The  computer  matched  the  requirements  of 
each  objective  with  the  capabilities  of  each  medium,  yielding  for 
each  objective  a  prioritized  list  of  canadidate  media. 


3.7  Step  7:  Determine  Alternate  Media  Assignment  Patterns 

At  this  time,  the  team  determined  six  different  media  mixes 
for  comparison.  As  a  preliminary  to  this  step,  a  complete  list¬ 
ing  was  run  comparing  the  instructional  requirements  for  each 
individual  objective  to  the  instructional  capabilities  of  each  of 
the  media  in  the  initial  media  pool.  This  resulted  in  a  priori¬ 
tized  list  of  all  media  which  met  minimal  instructional  require¬ 
ments  for  the  objective.  This  list  in  once  sense  represents  a 
seventh  media  pattern,  albeit  not  a  realistic  one.  On  the  basis 
of  an  analysis  of  the  constraints  generated  by  the  training 
support  requirements  analysis  process,  including  but  not  limited 
to  the  constraints  listed  in  Sections  2,  3,  and  2.4,  six  differ¬ 
ent  alternative  media  mixes  were  selected  for  detailed  comparison 
on  the  basis  of  instructional  effectiveness,  acquisition  cost, 
maintenance  cost,  instructional  motivation  characteristics, 
standardization,  and  special  requirements.  These  media  mixes  are 
listed  in  Table  7. 
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Table  7--Media  patterns  and  segments  selected. 

Pattern  1 


Slides/Audio  and  worksheet  139 

Training  manuals  and  worksheet  93 

Lecture  and  worksheet  93 

Lecture  and  worksheet  and  videotape  93 

Lecture  and  worksheet  and  (model)  equipment  278 

Pattern  2 

Slides/Audio  and  worksheet  10 

Slides  and  worksheet  142 

Workbook  72 

Lecture  and  worksheet  332 

Lecture  and  worksheet  and  videotape  70 

Lecture  and  worksheet  and  (model)  equipment  70 

Pattern  3 

CAI  and  part-task  trainer  and  worksheet  208 

Workbook  70 

Training  manuals  and  worksheet  348 

Lecture  and  worksheet  70 

Pattern  4 

CAI  and  part-task  trainer  and  worksheet  348 

Workbook  2^8 

Lecture  and  worksheet  and  student  response  70 

system  and  (model)  equipment 


Pattern  5 

Lecture  and  worksheet  and  student  response  896 

system  and  (model)  equipment 

Pattern  6 

CAI  and  part-task  trainer  and  worksheet  696 

Hie  six  media  mixes  were  selected  on  the  basis  of  the 
following  rationale.  The  first  media  mix  was  selected  because  it 
represents  the  most  common  mix  of  media  in  many  Air  Force  train¬ 
ing  settings  at  the  present  time.  Notice  that  this  represents 
five  different  general  media  approaches,  no  one  of  which  is  a 
single  medium.  For  example,  the  tape/slide  presentations  woul 
be  augmented  and  supported  by  a  worksheet  which  might  contain 
additional  directions,  test  items,  errata,  and  updates  to  the 
current  version  of  the  tape/slide  program,  and  so  on.  In  other 
medium,  the  worksheet  might  contain  photog raphs  ,  diagrams,  and 
other  information  to  support  the  lecture  or  primary  medium 


chosen.  It  is  Celt  that  this  is  a  more  realistic  approach  to 
media  designation,  since  very  few  media  in  their  "pure"  form 
totally  address  a  significant  range  of  instructional  situations. 
That  is,  most  instruction  really  requires  a  "multimedia" 
approach,  usually  to  teach  one  individual  objective. 

The  second  media  mix  represents  what  the  contractor  would 
v'h.UfU'terize  as  a  low  technology  media  mix,  with  one  important 
addition  to  the  commonly  encountered  Air  Force  media  mix  outlined 
in  media  pattern  one.  This  media  mix  adds  one  significant  item, 
a  formatted  instructional  workbook,  to  the  previous  media 
pattern.  As  will  be  seen  in  the  results,  this  one  addition  makes 
considerable  improvement  in  the  number  of  instructional  objec¬ 
tives  which  could  be  acceptably  addressed  by  the  resulting  media 
mix. 

The  third  media  mix  represents  a  higher  technology  approach 
with  characteristics  of  easy,  economical,  and  short  turnaround 
time,  update,  and  revision  capabi 1 i t i es .  It  consists  of  a  CAI 
system  supported  by  some  on-line  part-task  trainer  applications, 
and/or  hardware,  and  the  worksheet.  It  also  includes  formatted 
workbooks,  some  use  of  training/reference/maintenance  manuals 
supported  by  instructional  worksheets,  and  it  includes  lectures 
supported  by  student  handouts  (worksheets). 

Three  major  CAI  systems  were  considered  with  three  alternate 
configurations  of  each  of  the  TICCIT  and  PLATO  systems.  All  CAI 
systems  are  extremely  capable  and  address  the  majority  of 
instructional  objectives  very  well  ( i  .  e .  ,  in  an  unconstrained 
setting  they  are  usually  the  first  or  second  choice  for  most 
cognitive  objectives).  For  a  variety  of  reasons,  the  CAI  system 
recommended  in  these  media  mix  explorations  is  the  TICCIT  system. 

The  GETS  system,  in  its  current  configuration,  meets  most  of 
the  display  requirements  but  the  sophistication  and  economy  of 
its  authoring  system  is  suspect.  Since  it  is  designed  to  operate 
primarily  in  a  stand-alone  single  terminal  mode,  central  collec¬ 
tion  of  student  data  and  other  advantages  of  a  centralized 
terminal  network  device  are  absent.  In  addition,  since  it  is 
primarily  a  microprocessor  based  system,  at  the  current  state  of 
technology,  its  raw  computer  power  is  also  suspect. 

PLATO,  on  the  other  hand,  is  an  extremely  sophisticated 
system  with  generally  good  display  characteristics.  It  has  the 
most  computer  power  of  any  of  the  systems  compared,  and  its 
authoring  system  is  sophisticated  and  could  be  economically 
utilized.  Its  major  problem,  at  its  current  state  of  technology, 
is  that  it  is  an  extremely  expensive  system  in  any  configuration 
where  a  relatively  low  number  (30  or  40)  of  terminals  are 
required  in  one  site,  unless  the  terminals  are  tel eprocessed  to  a 
remote  central  processing  unit.  To  the  extent  that  a  consider¬ 
able  amount  of  the  current  F-lfi  academic  curriculum  (and  it  is 
assumed  of  any  future  revisions  and  updates)  is  classified, 
teleprocessing  to  a  remote  site  is  probably  not  a  viable 
alternative. 
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Both  PLATO  and  GETS  are  evolving  systems  as  is  TICCIT,  and 
in  the  event  any  decision  is  made  to  utilize  state-of-the  art 
media  technology  in  subsequent  revisions,  updates,  and  modifica¬ 
tions  to  the  F-16  syllabus,  the  CAI  systems  should  be  reevalu¬ 
ated.  The  current  recommendation  of  the  contractor  for  the  F-16 
setting,  is  that  the  TICCIT  system  is  the  system  of  choice. 

The  fourth  media  mix  eliminates  use  of  tra i ni ng/ ref er ence 
manuals  with  no  decrease  in  the  number  of  objectives  adequately 
addressed,  and  with  a  probable  considerable  increase  in  instruc¬ 
tional  effectiveness.  The  lecture  format  is  upgraded  by  the 
addition  of  a  student  response  system  (the  TICCIT  system  could  be 
used  for  this  also)  and  some  actual  (or  models  of)  F-16 
equipment . 

The  fifth  media  mix  represents  a  low  technology  approach 
utilizing  primarily  lecture  and  worksheet  supported  by  a  simp1 e 
student  response  system  and  models  of  equipment.  No  audiovisual 
technology  such  as  slides,  videotapes,  or  audiotapes  are  assumed. 
As  will  be  seen,  this  approach  fails  to  address  the  majority  of 
instructional  objectives  adequately.  In  addition,  it  would 
provide  very  little  variety  to  the  unfortunate  student. 

The  sixth  media  mix  represents  a  high  technology,  single 
media  exploration.  The  media  chosen  was  CAI  with  part-task 
trainer  capability  on  some  terminals,  supported  by  worksheets. 
This  media  mix  addresses  essentially  all  of  the  cognitive  objec¬ 
tives  in  the  F-16  syllabus  very  effectively.  The  richness  of 
this  Media  would  also  provide  a  surprising,  and  probably  more 
than  adequate,  amount  of  variety  to  the  student.  For  example, 
TICCIT's  color,  motion,  audio,  and  interactive  characteristics 
make  possible  a  wide  variety  of  stategies  and  formats,  and  really 
constitute  a  multimedia  approach  in  as  much  as  TICCIT  assumes  the 
capability  of  implementing  videotapes  on-line.  The  part-task 
trainer  may  be  either  2-d imens i ona 1  (2-D)  interactive  cathode  ray 
tube  (CRT)  displays  or  computer  driven,  3-dimensional  (3-D) 
equipment  attached  to  the  terminal. 

The  seventh  media  mix  consists  of  the  prim’ rive  media  selec¬ 
tion  data,  which  lists  all  of  the  media  in  the  overall  media  pool 
and  gives  the  data  on  each  of  the  media  as  if  it  were  the  only 
medium  allowed.  This  table  tells  how  many  of  the  696  objectives 
would  have  been  a  "primary  hit"  (i.e.,  out  of  all  the  media,  this 
medium  would  have  been  the  first  choice  for  the  objective)  ,  how 
many  were  addressed  by  this  medium  as  a  "secondary  hit"  (i.e., 
for  those  objectives,  this  medium  would  have  been  one  of  the  very 
top  choices,  but  not  the  first  choice) ,  how  many  were  addressed 
as  acceptable  by  the  medium  as  a  "minor  hit,"  (i.e.,  the  medium 
could  address  the  needs  of  the  objective,  but  it  is  by  no  means  a 
preferred  medium  to  use),  and  finally,  for  how  many  objectives 
was  that  medium  completely  unsatisfactory  in  one  or  more  of  the 
display,  response,  evaluation,  or  feedback  categories  determined 
to  be  important  for  the  objective. 
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3.8  Step  8:  Determine  Media  Production  and  Maintenance  Cost 

The  effort  involved  in  this  section  was  concentrated  primar¬ 
ily  upon  determining  the  initial  development  and  production  and 
life  cycle  revision  and  update  costs  to  the  curriculum  materials 
based  on  the  requirements  of  the  different  media  selected.  Table 
8  shows  the  cost  of  developing  and  producing  a  single  medium 
module,  or  segment  of  instruction,  in  each  media  class  for  each 
of  the  six  media  mixes  being  considered.  These  figures  are  based 
on  costs  incurred  in  past  materials  development  efforts  as 
recorded  by  the  contractor  and  include  personnel,  equipment,  and 
supply,  but  not  facility  costs.  This  table  also  shows  the  cost 
of  revising  or  updating  and  reproducing  a  single  medium  module  or 
segment  of  instruction,  for  each  of  the  media  in  each  of  the 
mixes.  Later,  this  data  will  be  combined  with  equipment  acquisi¬ 
tion  and  maintenance  costs  and  all  other  relevant  costs  taken 
from  the  training  support  requirements  analysis  and  reported  in 
Step  12  in  this  document  to  show  the  total  cost  of  the  training 
system,  given  each  media  assignment  pattern. 

Table  8--Cost  to  develop,  produce,  and 
revise  a  single  medium  module. 

Cost  to  Cost  to 

Media  Class _ Develop/ Produce _ Update/Revi  se 

CAI  (TICCIT)  1,004.90  248.36 

Tape/Slide  2,779.08  1,629.54 

Lecture  1,751.48  751.46 

Study  Guide  for  Manual  672.50  178.28 

Workbook  1,515.68 _ 422  .70 


3.9  Step  9:  Construct  the  Media  Decision  Models 

For  each  mix,  a  media  decision  model  was  constructed.  The 
primary  factors  to  be  considered  in  making  each  media  decision 
were  instructional  effectiveness,  cost,  student  motivation,  and 
ready  availability  of  standardized  implementation  and  support 
equipment.  As  has  been  reported  earlier,  the  selection  team  had 
at  its  disposal  a  complete  listing  of  the  relative  instructional 
effectivness  of  every  medium  in  the  entire  media  pool  on  each 
instructional  objective.  In  addition,  the  team  had  at  its 
disposal  a  more  focused  comparison  between  only  those  media  in 
the  media  mix  for  each  media  selection  pattern,  again  based  on 
instructional  effectiveness.  For  Step  7  of  the  media  selection 
process,  the  team  had  at  its  disposal  the  relative  development, 
production,  and  revision  costs  of  a  typical  module  in  each 
medium.  The  team  drew  upon  its  own  extensive  experience  in 
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military  aircrew  training  in  determining  the  relative  motiva¬ 
tional  characteristics  of  each  medium,  and  in  estimating  the 
requirements  for  variety. 

Finally,  the  initial  media  pool  was  determined  to  consist 
almost  entirely  of  standardized  equipment  and  approaches.  It  was 
assumed  that  in  every  case  any  desired  equipment  or  materials 
would  be  readily  available  from,  and  supported  by,  commercial 
vendors  from  off-the-shelf  stocks  or  from  standard  production 
runs.  Weighing  these  characteristics  against  the  intended 
emphasis  of  each  media  selection  pattern  (i.e.,  low  technology 
pattern  vs.  high  technology  pattern,  low  equipment  acquisition 
vs.  low  update  and  revision  costs,  etc.)  ,  the  total  number  of 
instructional  modules  to  be  assigned  to  each  medium  within  each 
pattern  was  determined.  Table  7  shows  the  number  of  objectives, 
from  the  696  objective  total,  assigned  to  each  medium  within  each 
media  selection  pattern. 

Figure  3  shows  the  general  media  selection  model  utilized 
within  each  pattern  to  make  the  individual  objective  media 
selections  within  the  constraints  of  the  preliminary  media  mix 
apportionment  . 


3.10  Step  10:  Select  Media  for  All  Objectives  under  Each 
Assignment  Pattern 

At  this  point  media  selection  proceeded  for  each  objective 
under  each  of  the  six  assignment  patterns.  The  computer i za t i on 
of  the  media  assignment  process  allowed  each  alternative  to  be 
computed  separately.  Each  objective  was  passed  through  the  logic 
of  each  assignment  pattern  and  assigned  an  alternative  medium. 
These  patterns  can  then  be  costed,  in  Step  9,  to  determine  the 
relative  overall  costs  of  the  alternate  media  patterns. 


3.11  Step  11:  Determine  the  Total  Cost  of  Each  Media  Assignment 
Pattern 

Using  the  cost  figures  for  each  media  pattern  derived  in 
Step  7,  and  the  appropriate  costs  taken  from  the  training  support 
requirements  analysis  document,  report  no.  21  "F-16  Instructional 
System  Cost  Study  Report,"  the  overall  cost  for  media  acquisition 
and  the  ongoing  costs  per  year  for  the  total  training  system, 
including  materials,  facilities,  personnel,  equipment,  etc.,  were 
calculated.  These  are  reported  in  Table  9.  This  table  shows, 
for  each  media  pattern  selected,  the  acquisition  cost  for  the 
total  training  program,  including  instructional  materials,  facil¬ 
ities  and  equipment,  and  the  ongoing  costs  per  year  for  person¬ 
nel,  maintenance,  and  curriculum  update  and  revision.  (These 
costs  apply  to  one  training  site.  Subsequent  sites  will  be  less 
expensive  since  the  curriculum  materials  will  be  already  devel¬ 
oped.)  Also  shown  are  the  life  cycle  costs  of  each  media  pattern 
over  10  and  20  years.  As  can  be  seen,  there  is  little  signifi- 
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Figure  3--Media  Selection  Model 
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cant  difference  in  the  cost  of  these  training  programs  due  to 
academic  media  selection.  The  costs  of  the  aircraft  operation 
(but  not  acquisition)  and  simulator  operation  and  acquisition 
represent  such  a  high  proportion  of  the  total  training  system 
cost  that  the  training  system  costs  are  relatively  insensitive  to 
costs  related  directly  to  academic  media. 

These  cost  figures  unavoidably  reflect  several  systematic 
biases.  They  inadequately  reflect  the  serious  consequences  of 
inflation  on  personnel  costs.  In  addition,  these  costs  seriously 
overestimate  the  cost  of  CAI  and  computer  based  part-task  trainer 
media.  For  nearly  30  years,  in  spite  of  inflationary  trends,  the 
dollor  cost  per  unit  of  work  done  by  computer  based  devices  has 
been  dramatically  and  steadily  coming  down,  unlike  personnel 
costs.  At  the  same  time,  power,  flexibility,  and  capability  are 
dramatically  increasing.  To  the  extent  that  personnel  costs  have 
been  underestimated  and  computer  costs  overestimated,  this  may 
seriously  distort  the  10  and  20  year  life  cycle  figures.  As  will 
be  seen  in  the  next  section  these  may  be  important  bias  factors 
to  bear  in  mind  since  there  are  serious  cost  trade-off  factors  to 
be  considered  in  the  F-16  training  scenario. 


3.12  Step  12:  Review  Cost  Tradeoffs  Related  to  Each  Assignment 
Pattern 

The  positive  and  negative  factors  listed  for  each  media 
assignment  pattern  in  Step  6  and  attached  to  i t  were  examined 
along  with  the  costs  derived  in  Step  10.  It  was  possible  to  con¬ 
duct  an  informal  cost  tradeoff  study  at  this  time.  The  developer 
chose  that  media  configuration  for  recommendation  which  presented 
the  best  set  of  positive  instructional,  cost,  and  use  features. 

No  set  of  guidelines  was  or  should  be  set  down  for  this  because 
it  involves  a  decision-making  process  on  the  part  of  the 
developer  and  the  instructional  system  consumer  which  may  involve 
many  factors  external  to  the  system  itself. 

By  producing  the  media  assignments  and  the  cost  data  attend¬ 
ant  to  each  for  several  assignment  patterns,  the  developer  has 
laid  the  foundation  for  orderly  decision-making  and  comparison  of 
several  ways  to  proceed.  Further  assignment  patterns  may  be 
generated  at  this  point  to  test  new  possibilities  for  media 
mixtures,  and  it  is  assumed  that  a  negotiation  between  the  devel¬ 
oper/consumer  and  the  real-world  contingencies  will  take  place  at 
this  time  until  the  best  combination  of  media  for  the  funds 
available  is  determined.  Moreover,  when  the  decision  is  reached, 
both  developer  and  consumer  have  an  understanding  of  the 
capabilities  and  limitations  of  the  resulting  media  system. 

TVie  media  factors  identified  in  Step  6  were  compared  to  each 
other  in  terms  of  overall  cost,  estimated  instructional  effec¬ 
tiveness,  projected  motivational  cha r ac t e r i st i cs  ,  and  on  the 
basis  of  each  of  the  11  critical  needs  of  the  F-16  training 
program  identified  in  the  analysis  of  the  F-16  training  manage¬ 
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ment  system  requirements  and  reported  in  the  "Management  System 
Needs  and  Design  Concep:  Analysis,"  development  report  no.  12, 
December  1977.  Given  that  computer  based  operational  flight 
trainers  (OFT)  were  to  be  available,  no  special  or  environmental 
concerns  beyond  those  already  imposed  by  the  OFT  were  identified. 

Also  taken  into  account  were  the  findings  of  the  computer 
managed  instruction  study  reported  in  "Computer  Managed  Instruc¬ 
tion  for  the  F-l 6  Training  Program,"  development  report  no.  17, 
July  1978.  The  supporting  rationale  for  utilizing  computer 
managed  instruction  in  the  F-16  training  program  was  clearly 
outlined  in  that  document.  The  11  critical  needs  outlined  in  the 
"Management  System  Needs  and  Design  Concept  Analysis"  paper  are 
equally  germane  inasmuch  as  they  are  needs  of  the  overall  F-16 
training  program,  and  not  just  needs  related  to  the  management  of 
that  program. 

The  approach  taken  in  this  section  of  the  media  selection 
document  was  to  rank  each  of  the  selected  media  patterns  on 
costs,  instructional  effectiveness,  motivation,  critical  training 
program  needs,  and  major  computer  managed  instruction  (CMI) 
subfunctions  (scheduling,  inventory,  and  instructional  prescrip¬ 
tions).  Multiple  rankings  were  obtained  for  each  of  the  six 
media  selection  patterns,  with  a  brief  supporting  rationale  for 
each  ranking.  Caution  must  be  taken  in  the  interpretation  of 
these  rankings  inasmuch  as  this  is  a  qualitative,  not 
quantitative  judgment,  and  for  any  given  set  of  circumstances  or 
analysis  perspective,  the  weighting  of  the  rankings  will  be  far 
from  equal.  On  the  basis  of  the  cost  and  other  data,  the  Air 
Force  may  decide  to  periodically  generate  new  media  mixes  and 
explore  their  capabilities  a  ,  new  information  becomes  available 
throughout  the  remainder  of  the  project.  This  process  is  greatly 
simplified  by  virtue  of  tne  fact  that  the  significant  data  base 
is  now  completely  automated,  along  with  a  powerful  set  of  compu¬ 
tational  program  tools,  for  their  manipulation  in  exploring  new 
alternatives.  On  the  basis  of  data  now  available,  and  the  judg¬ 
ment  of  the  media  selection  team,  the  following  cost  trade-offs 
pertain. 

3.12.1  Cost 

Cost  was  the  first  major  factor  to  be  considered.  Table  9 
has  already  reported  the  initial  acquisition  and  10-  and  20-  year 
life  cycle  total  cost  for  each  of  the  media  patterns.  As  can  be 
seen,  based  on  the  20  year  life  cycle  cost  (a  not  unreasonable 
time  period  in  the  life  of  a  modern  weapon  system)  only  a  9.0 
percent  difference  exists  between  the  bottom  line  costs  of  the 
most  expensive  and  least  expensive  media  pattern  alternatives. 
When  this  difference  is  weighed  against  the  fact  that  the  CAI 
supported  alternatives  also  offer  the  benefit  of  a  potential 
estimated  10  percent  reduction  in  simulator  time  required 
(conservative)  and,  when  one  considers  that,  as  has  been  pointed 
out  before,  the  cost  figures  are  biased  heavily  against  t  he 
computer  supported  media  patterns  on  at  least  two  different 
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important  factors,  this  9.6  percent  difference  becomes  less 
significant.  In  addition,  the  difference  between  the  pattern 
most  resembling  current  Air  Force  practice  and  the  pattern  which 
is  most  heavily  CAI/pa r t- task  trainer  supported,  is  less  than  2 
percent  in  total  life  cycle  cost,  even  without  taking  into 
account  the  anticomputer  bias  of  the  cost  figures.  For  a  direct 
cost  comparison,  the  numbers  will  be  allowed  to  speak  for  them¬ 
selves,  and  these  other  factors  will  be  taken  into  account  in 
later  rankings  . 

The  media  selection  patterns  were  given  the  following  cost 
rankings  by  the  media  selection  team: 

Patterns  2  and  6  were  ranked  1  (best  or  lowest  cost) . 

Patterns  1,  3,  and  4  were  ranked  2. 

Pattern  5  was  ranked  3. 

3.12.2  Instructional  Effectiveness 

On  the  second  major  factor,  instructional  effectiveness,  the 
media  selection  team  utilized  the  primitive  media  selection  data 
which  showed  the  extent  to  which  each  media  mix  dealt  with  the 
696  instructional  objectives  effectively;  i.e.,  the  extent  with 
which  one  or  more  media  in  that  media  pattern  was  a  first  or 
second  choice  in  which  to  teach  that  objective  on  the  basis  of 
instructional  capabilities  of  the  entire,  unconstrained  media 
pool.  In  this  comparison,  as  can  be  seen  from  the  tables,  there 
were  wide  differences  revealed  in  the  estimated  instructional 
effectiveness  of  these  six  media  patterns.  The  media  selection 
team  assigned  the  following  rankings: 

Patterns  3,  4,  and  6  were  given  rankings  of  1. 

Pattern  1  was  given  a  ranking  of  ?. 

Pattern  2  was  given  a  ranking  of  5. 

Pattern  5  was  given  a  ranking  of  6. 

3.12.3  Motivation 

The  media  patterns  were  then  compared  in  terms  of  their 
motivational  characteristics.  Characteristics  which  greatly 
enhance  the  motivational  reaction  of  students  to  the  instruction 
by  virtue  of  the  medium  include,  but  are  not  limited  to,  the 
interactive  capability  of  the  media  (i.e.,  the  degree  to  which  it 
engages  the  student's  attention,  encourages  his  interaction,  and 
allows  his  control);  display  salience  (i.e.,  such  things  as 
color,  sound,  graphics,  blinking,  motion,  etc.);  and  display 
latitude  (i.e.,  the  extent  to  which  a  medium  can  assume  very 
different  display  characteristics,  ranging  from  all  text  to 
heavily  graphic  types  of  presentations,  etc.)  This  area  is 
highly  subjective,  and  considerable  disagreements  between  raters 
can  easily  be  generated. 

A  main  active  principle  seems  to  be  the  extent  to  which  the 
medium  can  provide  the  student  with  a  rich  and  varied  menu  of 


displays,  interactions,  and  so  on  in  the  instructional  setting 
witn  tne  media  pattern  to  be  analyzed.  On  the  basis  of  these 
concerns,  the  media  selection  team  ranked  the  media  on  motiva¬ 
tional  cha r ac te r i st i c s  as  follows: 

Pattern  4  was  given  a  1  ranking. 

Pattern  3  was  given  a  2  ranking. 

Pattern  6  was  given  a  3  ranking. 

Pattern  1  was  given  a  4  ranking. 

Pattern  2  was  given  a  5  ranking. 

Pattern  5  was  given  a  6  ranking. 

Note  that  it  was  considered  by  the  team  that  those  cases 
where  a  rich,  interactive  medium,  such  as  CAI,  was  mixed  with 
lecture,  were  by  far  the  most  motivational  media  patterns.  Also 
note  that  the  all-CAI  pattern  was  considered  to  be  more  motiva¬ 
tional  due  to  its  rich  display  character i st i cs  and  its  flexibil¬ 
ity  (including  "gaming”  capabi 1 i ties)  than  an  all-lecture  format 
such  as  pattern  5.  This  data  seems  to  be  upheld  by  data  on  such 
settings  as  the  Navy's  S-3A  aircrew  setting  where  Naval  flight 
officers  report  very  good  attitudes  towards  extensive  blocks  of 
all  CAI  instruction  (even  after  4-6  hour  time  periods) . 

3.12.4  Maintainability 

The  next  factor  to  be  considered  was  the  first  of  the  11 
critical  needs  identified  for  the  F-16  training  program,  namely, 
the  need  for  lifetime  maintenance  of  the  training  program. 

Several  factors  affect  the  "maintainabil i ty"  of  a  training 
program.  One  factor  is  the  cost,  the  ease,  the  flexibility,  and 
the  personnel  required  to  revise  and  update  the  course  in  the 
media  mix  selected.  Another  factor  is  the  ease  with  which 
critical  data,  providing  the  basis  for  the  change,  can  be 
collected,  analyzed,  and  utilized.  Yet  another  contributor  to 
this  factor  lies  in  the  extent  to  which  a  medium  can  or  cannot 
discipline,  structure,  or  support  the  quality  control  of  those 
revisions  and  updates  while  providing  an  historical  audit  trail 
to  the  various  generations  of  updates  which  have  been  created. 
Other  factors  include  such  concerns  as  the  fact  that  a  highly 
varied  media  mix  requires  many  different  technologies  and  a  wide 
variety  of  personnel  skills  and  talents  to  maintain  revision  and 
update  efforts. 

Another  factor  to  be  considered  was  the  fact  that  in  those 
cases  where  a  media  pattern  depends  heavily  upon  materials 
external  to  the  training  program  ( i  .e . ,  materials  developed  for 
other  purposes,  such  as  gun  camera,  videotapes,  maintenance 
materials  and  references,  and  other  technical  manuals)  ,  the 
training  program  has  inadequate  control  and  information  to 
successfully  revise  and  update  the  program  completely.  Changes 
in  these  external  materials  are  often  consolidated  and  controlled 
by  other  people  not  familiar  with  the  training  implications  of 
changes  which  they  make. 


One  last  consideration:  Historically,  all-lecture  courses 
have  been  among  the  most  difficult  to  maintain  and  update  in 
military  programs.  The  uniformity,  consistency,  and  quality 
control  of  the  instructional  approach  and  method  of  use  most 
rapidly  deteriorates  in  those  cases  where  an  all-lecture  or 
heavy-lecture  orientation  is  taken.  Control  in  such  cases  is  for 
the  most  part  surrendered  by  the  training  program  itself  and  of 
necessity,  assumed  by  the  individual  instructors  who  may  vary 
widely  in  motivation,  orientation,  skill,  and  knowledge,  or  for 
any  given  instructor,  based  on  health,  interest,  and  a  variety  of 
external  concerns.  Even  though  lecture  is  an  easily  maintainable 
medium  physically,  it  is  almost  impossible  to  maintain  integrity. 
On  the  basis  of  these  and  other  considerations,  the  media  selec¬ 
tion  team  assigned  the  following  rankings  on  this  factor: 

Media  pattern  6  was  assigned  a  1  ranking. 

Media  pattern  4  was  assigned  a  2  ranking. 

Media  pattern  1  was  assigned  a  3  ranking. 

Media  pattern  3  was  assigned  a  4  ranking. 

Media  pattern  2  was  assigned  a  5  ranking. 

Media  pattern  5  was  assigned  a  <S  ranking. 

3.12.5  Standardization 

The  next  major  factor  to  be  considered  was  the  need  to 
standardize  the  approach  to  modifying  the  training  system.  This 
factor  deals  with  the  degree  to  which  a  media  pattern  implicitly 
or  explicitly  offers  information  or  control  which  would  tend  to 
standardize  the  approach  to  modifying  the  instructional  system 
including  curriculum,  organization,  sequence,  etc.  In  the  mili¬ 
tary  setting,  with  its  frequent  personnel  turnover,  this  can  be  a 
real  problem.  The  CAI  related  media  mixes,  with  their  inter¬ 
active  authoring  systems,  data  requirements,  and  syntax  struc¬ 
tures  would  have  an  advantage  here.  They  would  likewise  have  an 
advantage  due  to  the  fact  that  they  offer  the  potential  for 
better  record  keeping,  data  analysis,  and  computational  support 
in  subsequent  decision  making  concerning  mod i f i ca t i ons  to  the 
training  system.  In  addition,  the  more  complex  the  media  mix, 
that  is,  the  more  media  which  are  involved,  the  more  standards 
which  would  have  to  be  evolved.  On  the  basis  of  these  and  other 
concerns,  the  media  selection  team  assigned  the  following 
rankings : 

Media  pattern  6  was  assigned  a  1  ranking. 

Media  pattern  4  was  assigned  a  2  ranking. 

Media  pattern  3  was  assigned  a  3  ranking. 

Media  pattern  5  was  assigned  a  4  ranking. 

Media  pattern  2  was  assigned  a  5  ranking. 

Media  pattern  1  was  assigned  a  6  ranking. 

3.12.6  Information  Availability 

The  next  major  factor  considered  was  the  need  to  explore  the 
effects  of  varying  program  parameters.  This  deals  with  the 


extent  to  which  use  of  a  particular  media  pattern  readily 
provides  information  for  analytical  support  to  program  management 
for  support  decision  making.  Obviously  the  computer  assisted 
instruction  supported  systems  would  be  advantageous  on  this 
factor  as  well  as  any  media  pattern  which  included  student 
responding  systems  (assumed  to  have  data  collection  capabili¬ 
ties).  The  media  selection  team  assigned  the  following  rankings 
on  this  f ac to r  : 

Media  pattern  6  was  ranked  1. 

Media  pattern  4  was  ranked  2. 

Media  pattern  3  was  ranked  3. 

Media  pattern  5  was  ranked  4. 

Media  pattern  2  was  ranked  5. 

Media  pattern  1  was  ranked  6. 

3.12.7  Response  to  Change  Requirements 

The  next  factor  to  be  examined  was  the  need  for  the  media 
pattern  to  support  quick  response.  The  need  for  quick  response 
overlaps  with  some  of  the  concerns  of  rapid  and  simple  revision 
and  update.  It  also  includes  the  ability  to  restructure,  or 
modify,  important  characteristics  of  the  training  program,  such 
as  developing  alternative  sequences,  extracting  and/or  restruc¬ 
turing  subsets  of  the  curriculum  to  meet  changing  student  popu¬ 
lation  requirements,  etc.  The  extent  to  which  the  curriculum  can 
be  modularized,  re identi f i ed  ,  and  recombined  supports  high 
rankings  on  this  factor. 

Stand-alone  media  packages  which  could  be  recombined,  and  of 
course  computer  assisted  instruction  modules  have  the  advantage 
on  this  factor.  Choice  of  lecture  instruction  usually  assumes 
that  a  group  of  students  large  enough  to  make  an  instructor's 
time  worthwhile  will  be  assembled  for  a  block  of  instruction  with 
the  attendant  implication  that  several  objectives  worthy  of 
instruction  will  be  covered,  and  therefore  must  be  documented  in 
the  lecture  guide.  Lectures,  therefore,  suffer  a  disadvantage  on 
this  factor,  in  spite  of  the  fact  that  it  could  be  said  that  the 
instructor  is  the  most  flexible  of  the  media,  being  able  to  think 
and  modify  instruction  on  his  feet.  However,  the  idiosyncratic, 
nonstandardi zed ,  uncontrolled  nature  of  this  capability  is  seen 
as  a  serious  weakness.  Again,  to  the  extent  that  a  media  pattern 
depends  upon  external  support  material,  such  as  training  and 
reference  manuals,  videotapes  or  other  audiovisual  materials 
developed  outside  the  training  program,  that  media  pattern 
suffers  a  severe  d i sad van  tag e  .  The  media  selection  team  assigned 
the  following  rankings  on  this  factor: 

Media  pattern  6  was  ranked  1. 

Media  pattern  4  was  ranked  2. 

Media  pattern  2  was  ranked  3. 

Media  pattern  3  was  ranked  4. 

Media  pattern  5  was  ranked  5. 

Media  pattern  ]  was  ranked  6. 


3.12.8  Problem  Detection 

The  next  critical  factor  involves  the  need  for  early  detec¬ 
tion  of  problems  within  the  training  program.  The  extent  to 
which  the  training  program  can  gather  student  performance  data 
and  relate  it  to  specific  instructional  modules  is  extremely 
important.  Sensitivity  to  potential  and  existing  problems  is 
necessary  to  provide  optimum  management  in  all  phases  of  train¬ 
ing.  Early  problem  identification  is  important  in  two  ways.  It 
makes  t^s  problem  easier  to  solve,  and  it  minimizes  the  chance 
for  pi  .-am  disruption.  Obviously,  tightly  controlled  test  and 
student  data  situations,  such  as  CAI  and  student  response 
systems,  offer  significant  support  in  meeting  this  need. 

There  is  some  question  as  to  whether  the  instructor  or  the 
computer  function  more  effectively  in  detecting  problems  in  the 
system.  While  the  intuitions  of  an  experienced  instructor  are 
powerful  and  often  defy  all  logic  in  their  insight,  at  least  two 
problems  exist  in  dependence  on  this  means  of  problem  identifica¬ 
tion.  One  is  the  limited  scope  of  the  instructor  in  terms  of 
relating  isolated  pieces  of  data  to  the  overall  program.  That 
is,  an  instructor,  immersed  in  the  numerous  problems  of  day-to- 
day  student  performance  analysis  for  many  students  on  many  objec¬ 
tives,  may  have  trouble  detecting  subtle,  but  extremely  important 
patterns  of  performance  deficiency. 

The  second  problem  lies  in  the  fact  that  individual  instruc¬ 
tors  will  vary  widely  in  terms  of  their  intuition  and  insight, 
and  in  terms  of  the  accuracy  of  their  conclusions  regarding  the 
nature,  locus,  and  causes  of  problems.  Again,  the  extent  to 
which  a  media  pattern  depends  on  external  material  outside  its 
control  for  instructional  purposes  causes  problems  on  this  vari¬ 
able.  The  media  selection  team  assigned  the  following  rankings 
on  thi s  var iable  : 

Media  pattern  6  has  a  1  ranking. 

Media  pattern  5  has  a  2  ranking. 

Media  pattern  4  has  a  3  ranking. 

Media  pattern  3  has  a  4  ranking. 

Media  pattern  2  has  a  5  ranking. 

Media  pattern  1  has  a  6  ranking. 

3.12.9  Response  to  New  Demand  Levels 

The  next  factor  involves  the  need  for  flexibility  to  meet 
changing  demands  on  the  program.  This  factor  deals  with  the 
ability  of  the  training  program,  within  the  media  pattern,  to  be 
changed  quickly  to  meet  changing  requirements  such  as  changing 
student  population  cha  r  ac  ter  i  s  t  i  cs  .  Also  important  for  this 
factor  are  such  concerns  as  the  simple  ability  of  the  program 
within  the  pattern  to  rapidly  replicate  itself  to  handle  many 
more  students  in  a  very  short  time.  Of  course,  training  programs 
heavily  dependent  upon  actual  instructors  will  be  seriously 
disadvantaged  by  this  type  of  program.  When  there  is  a  critical 
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need  for  many  highly  trained  pilots,  difficulty  of  obtaining 
qualified  pilots  to  increase  the  ranks  of  the  instructor  corps 
and  the  time  available  to  adequately  train  them  for  the  task  will 
be  severely  limited,  which  argues  against  this  media  pattern. 

Again,  flexibility  deals  with  such  other  concerns  as  the 
ability  to  expand  and  take  over  as  many  flight  and  simulator 
related  activities  as  possible  in  the  event  training  flights  and 
fuel  must  be  seriously  constrained.  Stand-alone  mediated 
instructional  packages  not  requiring  instructors  would  '..-ve  an 
advantageous  position  on  this  factor,  since  additional  r.  lia 
equipment  can  generally  be  obtained  in  a  very  short  time  and 
additional  copies  of  media  packages  can  generally  be  quickly 
obtained  in  an  emergency.  On  the  basis  of  these  and  other 
concerns,  the  media  selection  team  assigned  the  following 
rankings  on  this  factor: 

Media  pattern  6  was  given  a  1  ranking. 

Media  pattern  4  was  given  a  2  ranking. 

Media  pattern  2  was  given  a  2  ranking. 

Media  Pattern  2  was  given  a  4  ranking. 

Media  Pattern  1  was  given  a  S  ranking. 

Media  pattern  5  was  given  a  A  ranking. 

3.12.10  Self-monitoring 

The  next  factor  is  the  need  for  a  self-monitoring  system. 

An  earlier  factor  dealt  with  the  ability  of  the  system  to  monitor 
student  performance  and  to  identify  early  problems.  This  factor 
deals  more  with  the  ability  of  the  media  pattern  to  provide 
information  concerning  its  own  success.  That  is,  does  the  media 
pattern  provide  data  collection  analysis  capability  that  would 
make  it  easier  to  determine  how  well  the  system  is  working  in 
terms  of  meeting  its  own  stated  performance  goals?  Is  there  an 
ability  to  screen  data  going  in  and  out  of  the  training  program 
to  detect  unreasonable  values?  How  is  the  training  program  doing 
compared  to  real  world  operating  goals,  etc.?  It  is  felt  that 
the  real  time  data  collection  and  analysis  capabilities  of  the 
CAI  and  student  responding  systems  have  an  advantage  on  this 
front;  however,  the  intuitions  and  insights  of  an  experienced 
instructor  are  not  to  be  discounted.  Clearly,  in  this  case,  the 
patterns  suffering  the  most  disadvantage  are  those  which  have 
heavy  emphasis  on  stand-alone  mediated  instructional  packages 
such  as  tape/slides,  workbooks,  reference  manuals,  etc.  On  the 
basis  of  these,  and  other  concerns,  the  media  selection  team  made 
the  following  rankings  for  this  factor: 

Media  pattern  f,  was  ranked  1. 

Media  pattern  4  was  ranked  2. 

Media  pattern  1  was  ranked  2. 

Media  pattern  3  was  ranked  4. 

Media  pattern  2.  was  ranked  r> . 

Media  pattern  1  was  ranked  6. 
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3.12.11  Data  Management 


The  next  identified  critical  needs  are  the  needs  to  effec¬ 
tively  deal  with  large  amounts  of  data  and  the  need  to  establish 
■  historical  data  base.  These  needs  can  actually  be  considered 
to  be  different  aspects  of  one  important  factor  which  has  been 
adequately  dealt  with  as  several  parts  of  the  other  factors  pre¬ 
viously  ranked,  such  as,  need  for  a  self-monitoring  system,  need 
for  quick  response,  need  to  explore  the  effect  of  varying  program 
parameters,  etc.  Therefore,  no  further  ranking  was  given  on 
these  factors  by  the  media  selection  team. 

3.12.12  Redundancy 

The  next  factor,  need  for  redundancy,  is  considered  to  be  an 
extremely  important  factor.  Redundancy  prevents  losing  important 
data  through  unforeseen  circumstances  and  deals  with  such  factors 
as  the  ability  of  the  system  data  collection  mechanism  to  detect 
and  protect  against  human  and  other  errors.  Loss  of  critical 
data  in  an  ongoing  program  greatly  reduces  effectiveness,  pre¬ 
vents  reliable  decision  making,  and  weakens  future  planning 
efforts.  Redundancy  deals  with  such  concerns  as  the  ability  of 
the  system  to  recover  from  equipment  downtime,  and  the  degree  to 
which  backups  are  available  (such  as  extra  tape/slide  projectors, 
etc  .)  . 


On  this  factor,  it  was  determined  that  1 ec t ur e- ba sed  media 
patterns  enjoyed  an  advantage.  Generally,  in  nonemergency 
situations  and  especially  within  TAC ,  there  will  be  extra 
academic  instructors  available  from  the  ranks  of  the  flight 
instructors  and  from  other  places.  Modern  computer  assisted 
instruction  systems,  with  their  extreme  reliability,  and  in  the 
case  of  TICCIT,  at  least,  the  ability  to  gradually  degrade  {  i  .  e  .  , 
pieces  of  the  system  can  break  down  without  bringing  down  the 
system) ,  also  are  strong  on  this  factor.  The  weakest  patterns  on 
this  factor  would  be  those  which  depend  on  hard  to  get, 
nonstanda rd i zed  ,  unreliable  equipment  in  short  supply.  Since 
care  was  taken  to  eliminate  such  media  devices  from  the  master 
media  pool  to  start  with,  it  was  felt  by  the  media  selection  team 
that  there  was  not  much  difference  between  the  media  patterns  on 
this  factor.  Their  ranking  on  this  factor  is: 

Media  pattern  5  was  given  a  1  ranking. 

All  other  media  patterns  were  given  a  ?  ranking. 

3.12.13  Operability  By  Users 

The  next  factor  to  be  considered  was  the  need  for  t he  system 
to  be  operable  at  the  user's  level.  That  is,  it  is  important 
that  the  training  media  employed  be  sufficiently  "human  engi¬ 
neered"  and  responsive  in  its  operation  that  users  with  little  or 
no  training  can  make  effective  use  of  the  media  and  materials 
with  little  or  no  instruction.  Highly  familiar  media,  such  as 
formatted  workbooks,  would  clearly  be  strong  on  this  factor. 
Lecture  based  patterns  would  similarly  be  strong. 


The  computer  assisted  instruction  patterns  had  the  potential 
to  be  even  stronger  on  this  factor  inasmuch  as  it  has  been 
demonstrated  that  elementary  school  children  can  make  very 
effective  use  of  computer  assisted  instruction  systems  if  care  is 
taken  in  the  proper  construction  of  the  lesson  materials,  the 
terminal  training  materials,  and  the  diagnostic  messages  given  to 
the  student  and  the  instructor.  Probably  the  weakest  on  this 
pattern,  are  some  of  the  so-called  "standard"  media  devices  such 
as  slide  projectors,  tape  recorders,  videotape  machines,  and 
student  responding  systems.  On  the  basis  of  these,  and  other 
considerations,  the  media  selection  team  assigned  the  following 
rankings  on  this  factor: 

Media  pattern  5  was  given  a  1  ranking. 

Media  pattern  6  was  given  a  2  ranking. 

Media  pattern  4  was  given  a  3  ranking. 

Media  pattern  3  was  given  a  4  ranking. 

Media  pattern  1  was  given  a  5  ranking. 

Media  pattern  2  was  given  a  5  ranking. 

3.12.14  Scheduling  and  Inventory 

The  CMI  concerns  outlined  in  development  report  no.  17,  July 
1978,  center  primarily  around  the  need  for  a  testing  component, 
student  records  component,  scheduling  component,  inventory  compo¬ 
nent,  and  prescriptive  component.  Testing  and  student  records 
have  been  dealt  with  extensively  in  factors  previously  consid¬ 
ered.  Scheduling  and  inventory  requirements  and  capability  will 
vary  widely  between  training  media  patterns.  Factors  which 
affect  a  media  pattern's  ranking  in  this  area  include  such 
considerations  as  the  amount  of  complex  scheduling  required  by 
the  training  program  administrators  in  recovering  from  a  nissed 
session  (e.g.,  missing  a  lecture  means  perhaps  as  many  as  20 
people  need  to  be  rescheduled  around  their  other  time  demands 
such  as  flying,  simulators,  etc.)  and  around  such  concerns  as 
making  maximum  use  of  perhaps  limited  materials  or  equipment 
(such  as  tape/slide  projectors,  etc.).  On  the  other  hand,  an 
inventory  component  deals  with  the  capability  of  a  media  pattern 
to  provide  information  on  the  stocks  on  hand  of  various 
consumables,  the  location  of  and  responsibility  for  equipment  and 
materials,  and  the  maintenance  status  of  critical  equipment,  etc. 

As  can  be  seen,  a  lecture-only  media  pattern  requires  little 
inventory  support  but  causes  problems  of  scheduling  with  little 
support  for  solutions  in  the  event  a  lecture  for  some  reason 
cannot  be  held  at  its  originally  identified  time.  Those  patterns 
which  will  be  the  weakest  on  this  factor  are  those  which  require 
a  lot  of  consumable  material  or  individual  pieces  of  audiovisual 
equipment,  and/or  those  which  have  limited  scheduling  flexibi¬ 
lity.  On  the  basis  of  these  and  other  considerations,  the  media 
selection  team  have  assigned  the  following  rankings  on  this 
facto  r  : 
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Med  i  a 

pattern 

2 

wa  s 

g  iven 

a 

5 

ranking  . 

3.12.15  Diagnostics  and  prescription 

The  final  factor  to  be  considered  in  the  media  selection 
team  pattern  rankings  is  that  of  the  ability  within  the  media  mix 
to  diagnose  student  errors  and  provide  individual  prescriptions 
to  help  the  student  correct  his  problems.  To  the  extent  that 
class  sizes  can  be  kept  small  and  consistent,  and  to  the  extent 
that  student  contact  with  individual  instructors  can  be  maximized 
so  the  instructor  learns  to  recognize  individual  student  problems 
through  extensive  interaction  with  the  same  students,  lecture- 
based  instruction  might  be  fair  on  this  factor. 

Certainly,  computer  assisted  instruction,  with  its  ability 
to  analyze  student  responses  and  recognize  many  alternatives,  to 
correct  incorrect  responses,  to  detect  time  taken  to  respond,  to 
record  every  student  response  and  analyze  student  performance  on 
the  basis  of  an  entire  performance  history  instead  of  one  or  two 
recent  answers,  would  be  very  strong  on  this  factor.  To  the 
extent  that  testing  done  on  student  responding  systems  were 
cleverly  and  carefully  designed,  and  to  the  extent  that  data 
analysis  capabilities  were  provided  with  the  student  responding 
systems,  they  would  be  reasonably  strong  on  this  factor. 

Probably  the  weakest  patterns  on  this  factor  are  those  where  the 
student  must  analyze  his  own  answers  and  determine  whether  he  was 
correct  or  incorrect.  On  the  basis  of  these  and  other 
considerations ,  the  media  selection  team  assigned  the  following 
rankings  on  this  factor: 


Media  pattern 
Media  pattern 
Media  pattern 
Media  pattern 
Media  pattern 
Media  pattern 


6  was  given  a  1  ranking. 
5  was  given  a  2  ranking. 
4  was  given  a  3  ranking. 
3  was  given  a  4  ranking. 
2  was  given  a  5  ranking. 
1  was  given  a  6  ranking. 


For  purposes  of  some  rough  comparisons,  and  in  recognition 
of  the  fact  that  the  average  reader  will  be  unable  to  resist  the 
temptation  to  do  so,  the  media  selection  team  has  provided  a 
summation  of  these  rankings  for  each  media  pattern.  Note  that  in 
any  given  situation,  any  one  of  these  14  factors  may  totally 
override  all  others  in  weight.  Therefore,  bear  in  mind  that  this 
summary  is  a  weak  general i zat ion  at  best,  and  is  not  to  be 
applied  blindly,  as  if  it  were  a  real,  quantitative  measure  of 
some  sort.  It  is  a  guide  to  thinking,  and  an  indication  only. 

The  individual  rankings  and  the  sum  are  summarized  in  Table  in. 

As  will  be  seen  later,  the  analysis  which  has  gone  into  the 
determination  of  these  rankings  has  provided  a  clear  set  of 
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TABLE  10  -  MEDIA  PATTERN  COST  TRADE-OFF  RANKINGS 


Assumption  is  that  manual  revisions  will  be  converted  to  CAI . 


recommendations  concerning  the  media  pattern  which  the  Air  Force 
should  embrace  as  the  training  program  matures.  Bear  in  mind,  in 
interpreting  the  table  and  the  rankings,  that  a  low  ranking  means 
an  advantageous  media  pattern. 


3.13  Step  13:  Develop  Near  Term  and  Far  Term  Selections 

At  this  point  in  the  selection  process,  the  project  team 
deviated  from  the  original  media  selection  plan.  Because  the 
initial  objectives  for  the  B/C  course  had  been  developed  with 
inadequate  OTD  team  input,  the  media  selection  team  determined 
that  a  media  mix  recommendation  would  be  inappropriate  without 
first  conducting  additional  analyses.  Consequently,  the  OTD  team 
reviewed  and  revised  the  list  of  ojectives  and  published  an 
updated  inventory  of  objectives  in  the  form  of  a  segment  list. 
These  objectives  were  then  reentered  into  the  media  selection 
process  at  Step  6:  Determine  Ins t r uc t i ona 1  Media  Requirements  of 
Each  Objective.  Because  of  the  similarity  between  the  second 
objectives  list  and  the  first,  this  change  had  little  impact  on 
Steps  1-5.  Because  the  later  stages  of  media  selection  deal 
primarily  with  mixes  of  media  rather  than  objectives,  this  change 
had  little  impact  on  Steps  7-12.  Still,  it  represented  an  extra 
investment  of  resources  that  may  be  avoided  by  future  development 
teams,  and  an  unnecessary  recycling  through  the  media  selection 
process . 

The  media  selection  team  now  had  at  its  disposal  the  follow¬ 
ing  information:  A  prioritized  list  of  media  mix  alternatives 
(Table  10),  a  prioritized  list  of  academic  media  for  each 
academic  objective,  and  a  prioritized  list  of  simulation  media 
for  each  hands-on  objective.  Their  task  was  to  examine  the  media 
selections  for  each  objective,  objective  by  objective,  determin¬ 
ing  first,  second,  and  third  choice  media  for  the  near  term  1980 
syllabus  and  for  the  far  term  1990's  syllabus.  The  purpose  of 
selecting  second  and  third  options  was  to  allow  for  flexibility 
in  later  choices.  A  syllabus  consisting  of  all  first  choice 
media  could  be  extremely  unbalanced  in  its  response  to  the  14 
evaluation  factors  listed  in  Table  10.  Such  a  syllabus  could  be 
prohibitively  expensive,  bore  students  by  chaining  them  to  a 
single  medium  all  day,  or  distract  them  by  constantly  switching 
them  from  one  medium  to  another.  There  is  no  one  perfect  medium 
for  all  objectives.  Tradeoffs  among  the  evaluation  criteria  must 
be  negotiated  in  order  to  develop  a  syllabus  that  balances  the 
evaluation  criteria  against  one  another  as  well  as  against  the 
identified  constraints  of  the  instructional  system. 

The  media  selection  team  developed  two  media  mixes.  The 
first  mix  was  the  unconstrained,  far  term  ideal  syllabus.  The 
second  mix  was  the  real  world,  near  term  1980  syllabus.  These 
mixes  were  as  follows: 
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Table  ll--Far  Term  Syllabus  Academic  Media  Selections 


No  .  Of 

_ Med  i  um _ Objectives 

1.  Tutorial  +  lesson  guide  160 

2.  CAI  +  lesson  guide  109 

3.  Tu to  rial  +  visual  motion  41 

4.  CAI  +  videotape  34 

5.  Tutorial  +  model/actual  equipment  27 

6.  CAI  +  videotape  +  part-task  trainer  5 

7.  Workbook  +  slides  1 

8.  Lecture  +  lesson  guide  1 

9.  No  acceptable  medium  1 

TOTAL  379 


Table  12 — Near  Term  Syllabus  Academic  Media  Selections 


No  .  Of 

_ Med  i  um _ Objectives 

1.  Lecture  +  lesson  guide  108 

2.  Workbook  +  lesson  guide  98 

3.  Lecture  +  visual  motion  66 

4.  Seminar  +  lesson  guide  43 

5.  Lecture  +  model/actual  equipment  19 

6.  Workbook  +  slides  14 

7.  Lecture  +  student  response  system  13 

8.  Seminar  +  visual  motion  8 

9.  Seminar  +  model/actual  equipment  6 

10.  Lecture  +  audio  5 


Table  12  (Continued) 


No  .  Of 

_ Med  i  um _ Objectives 

11.  Seminar  +  audio  2 

12.  Workbook  +  audio  1 

13.  Tutorial  +  model/actual  equipment  1 

14.  CFT  +  tape/slides  1 

TOTAL  385 


Notice  that  the  number  of  objectives  for  these  two  syllabi 
differ  by  six.  Each  time  the  objective  list  was  reviewed,  objec¬ 
tives  were  altered  through  addition,  deletion,  and  modification. 
As  each  successive  iteration  used  newer  and  more  accurate  data, 
the  old  data  were  stored  in  the  historical  data  base. 

While  the  OTD  team  was  satisfied  with  the  far  term 
syllabus,  the  near  term  syllabus  media  selections  did  not  seem 
consistent  with  the  manpower  constraints  of  the  F-16  training 
program.  McDill  AFB  is  programmed  to  have  a  12  man  staff  of 
instructors,  but  since  the  squadron  commander  and  operations 
officer  will  probably  not  teach,  their  instructor  staff  is 
actually  only  10.  Ten  men  is  a  small  staff  for  a  media  mix  so 
heavily  dependent  on  instructor  intensive  media  such  as  seminars, 
tutorials,  and  lectures.  Therefore,  the  OTD  team  reviewed  their 
initial  media  selections  for  academic  objectives  and  produced  an 
alternative  near  term  syllabus  that  was  less  instructor  inten¬ 
sive  . 
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Table  13--Revised  Near  Term  Syllabus  Academic  Media  Selections 


No  -  Of 

_ Med  i  urn _ Objectives 

1.  Seminar  +  lesson  guide  157 

2.  Seminar  +  visual  motion  68 

3.  Workbook  +  slides  61 

4.  Workbook  +  lesson  guide  60 

5.  Seminar  +  model/actual  equipment  23 

6.  No  media  assigned  (oversight)  7 

7.  Tutorial  +  model/actual  equipment  5 

8.  Tape/slide  +  lesson  guide  3 

9.  Workbook  +  audio  3 

10.  Seminar  +  audio  2 

11.  Lecture  +  model/actual  equipment  2 

12.  Lecture  +  student  response  system  2 

13.  Videotape  +  lesson  guide  1 

14.  Tutorial  +  lesson  guide  1 

15.  Tutorial  +  visual  motion  1 

16.  No  acceptable  media  1 

TOTAL  397 


Hie  hands-on  or  simulation  media  selection  process  did  not 
require  a  rewrite  of  the  initial  list  of  objectives.  Again,  two 
syllabi  were  developed,  one  for  near  term  and  one  for  far  term. 
The  initial  selections  were  as  follows. 
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Table  14--Near  Term  Syllabus  Training  Device  Selections 


No .  Ins 
tional 


F-16  or  F-16B  aircraft  183 

CFT  101 

AS  PT  10  0 

DSS  80 

Actual  equipment  23 

SMS  trainer  22 

Avionics  display  21 

F-16B  9 

EPT  6 

Cockpit  mockup  3 

Panel  mockup  1 

F-16A  0 

WST  0 

SAAC  0 

RWR  0 

Stick  and  throttle  trainer  _ 0_ 

TOTAL  549 
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Table  15--Far  Term  Syllabus  Training  Device  Media  Selections 


No .  Instruc¬ 
tional  Events 


OFT  265 

F-16A  or  F-16B  aircraft  298 

CFT  80 

OFT  and  NVS  67 

F-16B  12 

SMS  trainer  9 

Actual  equipment  7 

EPT  6 

OFT  and  DRLMS  6 

AS  PT  0 

DSS  £ 

TOTAL  650 


Notice  that  the  number  of  instructional  events  is  greater 
for  the  far  term  syllabus.  This  represents  a  desire  on  the  part 
of  the  OTD  team  for  greater  instructional  flexibility  in  the 
syllabus  of  the  future.  There  are  currently  just  over  500 
hands-on  objectives.  Some  of  these  objectives  are  not  satisfied 
by  a  single  instructional  event.  In  a  single  training  device, 
however,  some  require  a  two  or  three  device  progression,  such  as 
CFT,  OFT,  F-16B.  This  requires  events  and  trainers  for  a  single 
objective.  This  system  is  explained  in  greater  detail  in 
development  report  no.  23,  "F-16  Instructional  Sequencing  Plan 
Report ." 


3.14  Step  14:  Prepare  Segment  List 

This  step  represents  a  stage  of  the  syllabus  development 
process,  but  also  has  considerable  impact  on  media  selection. 

The  instructional  objectives  were  organized  into  segments  or 
lessons  of  10-20  minutes  each,  consisting  of  3-5  objectives.  If 
these  objectives  were  in  different  media,  the  development  team 
had  to  select  the  single  medium  most  appropriate  to  that  segment. 
In  most  cases,  the  i nst r uct iona 1  segment  had  a  single  or  at  least 
a  dominant  medium,  so  the  decision  was  straightforward. 
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3.15  Step  15:  Update  Media  Selection 


Subsequent  development  steps,  such  as  segment  listing, 
lesson  planning,  lesson  modification,  and  syllabus  update  all 
impact  the  media  selected  for  use  in  the  F-16  training  program. 
Because  the  original  prioritized  list  of  media  selected  for  each 
objective  is  still  available  in  the  project  data  base,  second, 
third,  and  fourth  choice  media  are  readily  available.  Where  new 
objectives  are  developed,  the  development  team  can  go  directly  to 
step  6,  and  determine  instructional  media  requirements  for  each 
objective.  This  involves  completing  an  objectives  analysis 
worksheet  (Tables  1  and  2)  for  each  new  objective  and  comparing 
it  directly  to  a  media  selection  matrix  (Tables  3  and  4).  This 
will  produce  a  ranked  list  of  media  alternatives,  which  may  be 
selected  from  on  the  basis  of  ongoing  system  constraints,  as 
identified  in  Section  2.4. 
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4.0  LIST  OF  AUDIO  VISUAL  PROGRAMS 


This  list  is  the  original  version  (8  Aug  1979)  of  the  seg¬ 
ments  to  be  produced.  Revisions  are  made  to  this  list  affecting 
the  media  selection,  wording  of  the  objectives,  and  length  of  the 
programs.  These  revisions  are  made  in  a  systematic  process  of 
both  contractor  and  OTD  team  review  and  approval.  An  updated 
list  of  segments  is  available  at  any  time  TAC  may  request  one. 

The  following  is  a  list  of  tape/slides  and  workbook  slide 
type  programs  only.  For  a  complete  segment  list  including  all 
media  types  see  development  report  no.  24,  "F-16  Coursewares  and 
Syllabi  Delivery  Schedule." 


Es  t . 

Type  of  Time/ 

Program _ _ _ Title _ ___ _ Hours 


Tape  Slide 


TS 101 

Exterior  Aircraft  Inspection  Checklist  Pro¬ 
cedures 

.  5 

TS 10  2 

Normal  and  External  Power  Engine  Start  Pro¬ 
cedures 

.  5 

TS103 

Procedures  for  Single-Ship  Line-Up,  Takeoff, 
Climb,  and  Departure 

.  3 

TS  104 

HOM  mode:  Uses,  Procedures,  and  Symbology 

.  3 

TS  1 0  5 

EDR  and  RNG  Modes;  Uses,  Procedures,  and 
Symbology,  Including  Cruise  Energy  Manage¬ 
ment  Considerations 

.  4 

TS  106 

Introduction  to  Navigation  Aids 

.  3 

TS107 

Procedures  for  Formation  Taxi  and  Formation 

Li  neup  Checkl i st 

.  3 

TS  108 

Lost  Wingman  and  Chase  Procedures 

.  4 

TS110 

Auto  Pilot  System 

.  5 

TS11 1 

Malfunction  Iden t i f i ca t i on  :  Dual  FCS  Fail¬ 
ures 

.  5 

TS112 

Malfunction  Identification:  Flight  Control 

Fa  i  1  u  r  e  s 

.  5 

TS113 

Malfunction  Identification:  Engine  Fire/ 
Overhea  t 

.  5 
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Type  o  f 
Program 

Title 

Est  . 
Time/ 
Hours 

TS 114 

Malfunction  Identification:  Engine 

.  4 

TS 115 

Malfunction  Identification:  Fuel  System 

.  3 

TS  116 

Malfunction  Identification:  Electrical 

Power  Supply 

.  4 

TS  117 

Malfunction  Identification:  Hydraulic 

System 

.  4 

TS  20 1 

Missile  Attack  in  AAM  Mode  with  AIM-9J 

.  3 

TS202 

Missile  Attack  in  AAM  Mode  with  AIM-9L 

.  3 

TS203 

Switchology  and  HUD  Symbology  for  Back  Up 
Modes 

.  5 

TS  20  4 

Switchology  and  Symbology  for  AIM-9  Employ¬ 
ment  Using  Manual  Reticle 

.  5 

TS204 

Procedures  for  Air-to-Air  Radar  Search  and 
Lock-on 

.  5 

TS  206 

Day/Night  Interceptor  Signals 

.  3 

TS301 

Malfunction,  Contingencies,  and  Abnormal 
Procedures  During  Air-to-Air  Refueling 

.  4 

TS302 

HUD  Energy  Management  Symbology 

.  S 

TS601 

External  Stores  Inspections  Including  Wing 
and  Center  Pylons,  BRU  31/A,  and  SUU  20  B/A 

.  4 

TS602 

Procedures  for  Verifying  Position  Using  INS 
Data  and  Performing  All  INS  Updates  Includ¬ 
ing  Radar  Altitude  Calibration 

.  5 

TS603 

Airborne  Radar  Approach  Procedures 

.  4 

TS801 

Chaff/Flare  Preflight  and  Setup 

.  S 

Workbook 

Slide 

WS101 

Cockpit  Ingress,  Interior  and  Prestart 

Chec  kl i sts 

.  5 

WS102 

Considerations  and  Application  of  Flight 
Control  System  Sel f-test 

.  3 
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Type  o  f 
Program 

WS103 

WS 10  4 

WS105 

WS106 

WS107 

WS108 

WS109 

WS110 

WS111 

WS112 
WS11  3 
WS114 

WS115 

WS116 
WS1 1  7 

WS  1 1  8 

WS  1 1  9 

WS120 

WS121 

WS122 


Title 


Before  Taxi  Checklist  Procedures 

Taxi  and  Before  Takeoff  Checklists,  Includ¬ 
ing  Single-Ship  Taxi  and  Maintenance  Arming 
Procedures 

Takeoff  Emergency  Procedures  Including 
Abort,  Arrestment,  and  Low  Altitude  Ejection 

Procedures  for  Night  Ground  Operations 

Night  Visual  Distress  Signals 

Procedures  for  Formation  Departure 

Procedures  for  Flying  Formation 

Two -Ship  Tactical  Formation  Positions 

Prgcedures  for  Two-Ship  Delayed  90°  and 
45  Tactical  Turns 

Procedures  for  Formation  Recovery 

Four-Ship  Tactical  Formation  Overview 

Prgcedures  for  Four-Ship  Delayed  90°  and 
45°  Turns 

Local  Area  Terminal  Procedures  Including 
VFR  Reentry 

Penetration  and  Enroute  Descent  Procedures 

TACAN  Holding,  Penetration,  and  Approach; 
Circling  Approach;  and  Missed  Approach 

Procedures  for  ILS  and  Localizer  Only 
Approaches 

Procedures  for  PAR,  ASR,  and  Low  Altitude 
Bo  x  Pa  1 1  e  r  n 

Varieties  of  Approaches  (Visual  and  Instru¬ 
ment)  and  Applications  of  Each 

Procedure  for  INS  Alignment,  Including  Pre¬ 
sent  Position  and  Destination  Coordinates 

Procedures  for  Stored  Heading  and  Best 
Available  True  Heading  (BATH)  Alignment 


Est  . 
Time/ 
Hours 

.  3 

.  5 

.  5 

.  3 
.  3 
.  4 
.  5 
.  5 
.  5 

.  5 
.  5 
.  5 

.  5 

.  5 
.  5 

.  5 

.  5 

.  5 

.  5 

.  3 
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Type  o  f 
Program 

WS123 


Title 


Est  . 
Time/ 
Hours 


SMS  Procedures  for  Stores  Loading  and  .5 

Verification 


WS124 

Penetration  AIDS  System 

1 . 0 

WS201 

Switchology  and  Symbology  for  AIM-9L  in 

Missile  Override/Dogfight  Mode 

.  4 

WS202 

Special  Considerations  for  Missile  Attack 
in  Override/Dogfight  Mode 

.  4 

WS203 

Procedures  for  Formation  Takeoff  Including 
Flight  Lead  Considerations 

.  4 

WS204 

Procedures  and  Special  Considerations  for 
IMC/Night  Intercept  and  ID 

1 . 0 

WS301 

SMS  Setup,  Verification,  and  Loading:  An 
Overview 

.  5 

WS302 

AAR  Procedures:  Observation  Position,  For¬ 
mations,  Refueling  Sequences,  Aircraft 

Systems  Checks 

.  5 

WS303 

AAR  Procedures:  Precontact,  Contact,  Main¬ 
taining  Position,  Disconnect,  and  Radio  Calls 

.  5 

WS304 

Computed  Gun  Attack  Modes:  LCOS  and  SS 

.  5 

WS305 

Procedures  for  Formation  Recovery  and  Land- 

.  4 

ing  (Lead  and  Wing) 
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